Fatalities Reportedly Occur Despite VioGén Algorithm's Low or Negligible Risk Scores

July 18, 2024

The VioGén algorithm was designed to help Spanish police assess and prioritize the risk of repeat domestic violence incidents. However, its low-risk assessment of Lobna Hemid reportedly led to inadequate protection; her husband murdered her. Since 2007, 247 women have been killed after being assessed by VioGén. A review of 98 homicides found that 55 of the slain women were scored as negligible or low risk. Despite these alarming statistics, some experts argue that the algorithm's limitations and lack of human oversight contribute to its inability to accurately predict high-risk cases. This raises serious questions about the effectiveness of AI-powered systems in addressing domestic violence and highlights the need for further research and development into more robust and trustworthy AI models. And to learn more about AI incident reporting, visit JOIN US.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
Spanish law enforcement agencies, Spanish Interior Ministry, Law enforcement, Government of Spain
Alleged developer
VioGén algorithm development team, Spanish law enforcement agencies, Spanish Interior Ministry, Law enforcement, Government of Spain
Alleged harmed parties
Women and girls, Women, Victims of domestic abuse, Stefany González Escarraman, María, Luz, Lobna Hemid, General public of Spain, General public, Eva Jaular, 247 women in Spain (unnamed)

AI governance case studies

For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/747

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.