MAP 1.6

Objective

System requirements (e.g., “the system shall respect the privacy of its users”) are elicited from and understood by relevant AI actors. Design decisions take socio-technical implications into account to address AI risks.

Incidents linked to this control

From automatic embedding match (not a formal assessment).

Facial Recognition Program in São Paulo Metro Stations Suspended for Illegal and Disproportionate Violation of Citizens’ Right to Privacy

April 12, 2018

Argentinian City Government Deployed Teenage-Pregnancy Predictive Algorithm Using Invasive Demographic Data

April 11, 2018

All Image Captions Produced are Violent

April 2, 2018

Man Arrested For Sock Theft by False Facial Match Despite Alibi

February 15, 2018

Google Photo Merge Decapitates Subject

January 25, 2018

Customer Service Robot Scares Away Customers

January 22, 2018

Gmail’s Inbox Sorting System Reportedly Reduced Visibility of Political Emails and Campaign Calls-to-Action

January 15, 2018

Instagram's Exposure of Harmful Content Contributed to Teenage Girl’s Suicide

November 21, 2017

Robot Deployed by Animal Shelter to Patrol Sidewalks outside Its Office, Warding off Homeless People in San Francisco

November 15, 2017

Amazon Alexa Plays Loud Music when Owner is Away

November 9, 2017

iPhone Face ID Failed to Recognize Users’ Morning Faces

November 1, 2017

Russian Chatbot Supports Stalin and Violence

October 12, 2017