Underground Market for LLMs Powers Malware and Phishing Scams
December 1, 2023
A study by Indiana University researchers uncovered widespread misuse of large language models (LLMs) for cybercrime. Cybercriminals, according to that study, use LLMs like OpenAI's GPT-3.5 and GPT-4 to create malware, phishing scams, and scam websites. These models are available on underground markets, often bypassing safety checks through jailbreaking. Named malicious LLMs are BadGPT, XXXGPT, Evil-GPT, WormGPT, FraudGPT, BLACKHATGPT, EscapeGPT, DarkGPT, and WolfGPT.
- Alleged deployer
- cybercriminals, badgpt, xxxgpt, evil-gpt, wormgpt, fraudgpt, blackhatgpt, escapegpt, darkgpt, wolfgpt
- Alleged developer
- openai
- Alleged harmed parties
- internet-users, organizations, individuals-targeted-by-malware
Source
Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/736
Data source
Incident data is from the AI Incident Database (AIID).
When citing the database as a whole, please use:
McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.
Pre-print on arXiv · Database snapshots & citation guide
We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.