Algorithm Assessing Risk Faced by Victims of Gender Violence Misclassified Low-Risk Cases, Allegedly Leading to Homicide of Women and Children in Spain

July 26, 2007

In Spain, the VioGén algorithm, designed to assess recidivism risk in gender violence cases, has allegedly underestimated potential danger in several instances. Regrettably, these misclassifications have reportedly resulted in homicides of women and children since its initial deployment. This underscores the need for trustworthy AI practices and governance within Project Cerebellum's AI incident database to prevent such harm.

JOIN US to learn more about how you can contribute to safe and secure AI practices through the HISPI Project Cerebellum TAIM, particularly in areas of Govern for such critical applications.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

MEASURE 2.6 — similarity 0.736, rank 1. TAIM detail and related incidents →
MAP 4.1 — similarity 0.706, rank 2. TAIM detail and related incidents →
MEASURE 3.1 — similarity 0.695, rank 3. TAIM detail and related incidents →

Alleged deployer: spanish-ministry-of-interior
Alleged developer: spanish-secretary-of-state-for-security, spanish-ministry-of-interior
Alleged harmed parties: spanish-victims-of-gender-violence

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/186

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.