Unverified Misreading by Automated Plate Reader Led to Traffic Stop and Restraint of an Innocent Person at Gunpoint in California

March 30, 2009

In San Francisco, an automated license plate reader (ALPR) camera misread a number as belonging to a stolen vehicle having the wrong make, but its photo was not visually confirmed by the police due to poor quality. Allegedly, despite multiple chances prior to making a traffic stop, the police pulled over an innocent person at gunpoint and restrained them in handcuffed. This incident raises concerns about the reliability of automated systems in law enforcement and the importance of human oversight to prevent such misreadings.

This incident highlights the devastating consequences of relying solely on automated systems without proper human review, and it's essential that law enforcement agencies in California take immediate steps to address the issue and prevent similar misreadings from happening again. By implementing robust protocols for verifying visual confirmations and increasing transparency around ALPR use, we can work towards a safer and more just system for everyone. And to learn more about AI incident reporting, visit JOIN US.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
San Francisco Police Department
Alleged developer
Unknown
Alleged harmed parties
Denise Green

AI governance case studies

For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/245

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.