Sound Intelligence's Aggression Detector Misidentified Innocuous Sounds

June 25, 2019

The misidentification of innocuous sounds by Sound Intelligence's aggression detector raises fundamental questions about the accountability of AI systems in sensitive environments. As educators and policymakers grapple with the consequences of this incident, it is crucial that they prioritize transparent testing, robust evaluation, and strict governance to prevent similar false positives from occurring in schools again. This incident underscores the urgent need for evidence-based AI development and responsible deployment practices to ensure student safety without compromising academic freedom JOIN US.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
Rock Hill Schools, Pinecrest Academy Horizon
Alleged developer
Sound Intelligence
Alleged harmed parties
Students, Rock Hill School students, Pinecrest Academy Horizon students, Epistemic integrity, Educational communities

AI governance case studies

For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/404

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.