Google’s Automated Child Abuse Detection Wrongfully Flagged a Parent’s Naked Photo of His Child
August 21, 2022
Google’s automated detection of abusive images of children incorrectly flagged a parent’s photo intended for a healthcare provider, resulting in a false police report of child abuse, and loss of access to his online accounts and information.
- Alleged deployer
- Alleged developer
- Alleged harmed parties
- a-software-engineer-named-mark, parents-using-telemedicine-services
Source
Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/303
Data source
Incident data is from the AI Incident Database (AIID).
When citing the database as a whole, please use:
McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.
Pre-print on arXiv · Database snapshots & citation guide
We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.