Hallucinated Software Packages with Potential Malware Downloaded Thousands of Times by Developers
December 1, 2023
Generative AI hallucinated non-existent software packages, which were then created and uploaded (as an experiment) by security researcher Bar Lanyado. One such package, "huggingface-cli," was downloaded over 15,000 times, including by large companies like Alibaba. Regardless of the framing of it as an experiment, this incident is an example of harm caused by AI-generated hallucinations in coding, as the fake packages were still distributed widely and with potential malware.
- Alleged deployer
- developers-using-ai-generated-suggestions, bar-lanyado
- Alleged developer
- bar-lanyado
- Alleged harmed parties
- developers-and-businesses-incorporating-ai-suggested-packages, alibaba
Source
Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/731
Data source
Incident data is from the AI Incident Database (AIID).
When citing the database as a whole, please use:
McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.
Pre-print on arXiv · Database snapshots & citation guide
We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.