HISPI Project Cerebellum HISPI PROJECT CEREBELLUM

Whisper Speech-to-Text AI Reportedly Found to Create Violent Hallucinations

February 12, 2024

New research reveals that OpenAI's Whisper, a speech-to-text system, may generate violent hallucinations and fabricated details, particularly with prolonged pauses in speech. The study analyzed 13,000 clips, discovering 1% contained harmful hallucinations. Such errors can pose risks in hiring, legal trials, and medical documentation. This underscores the importance of enhancing model training to minimize these incidents for diverse speaking patterns. Moreover, this incident highlights the need for robust AI governance and safety measures, emphasizing the role of Project Cerebellum in shaping a safe and secure AI future. For those interested in driving such change, JOIN US.

To map this incident to HISPI Project Cerebellum TAIM, we can focus on Measuring the impact of these incidents on diverse communities and Managing strategies to minimize harm in AI applications.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
openai, whisper, companies-using-whisper, organizations-integrating-whisper-into-customer-service-systems
Alleged developer
openai
Alleged harmed parties
individuals-with-speech-impairments, users-whose-speech-is-misinterpreted-by-whisper, professionals-relying-on-accurate-transcriptions, general-public

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/732

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.