MAP 1.6

Objective

System requirements (e.g., “the system shall respect the privacy of its users”) are elicited from and understood by relevant AI actors. Design decisions take socio-technical implications into account to address AI risks.

Incidents linked to this control

From automatic embedding match (not a formal assessment).

Facial Recognition in Remote Learning Software Reportedly Failed to Recognize a Black Student’s Face

February 1, 2021

Amazon’s Search and Recommendation Algorithms Found by Auditors to Have Boosted Products That Contained Vaccine Misinformation

January 21, 2021

Facebook's Automated Moderation Mistakenly Flagged Landmark's Name as Offensive

January 15, 2021

Facebook Recommended Military Gear Ads Despite Pause on Weapons Accessories Ads

January 13, 2021

Israeli Tax Authority Reportedly Used an Opaque Automated System to Issue a Fine, Declining to Explain or Disclose the Underlying Calculation

January 11, 2021

Hawaii Police Deployed Robot Dog to Patrol a Homeless Encampment

January 10, 2021

XPeng Motors Fined For Illegal Collection of Consumers’ Faces Using Facial Recognition Cameras

January 1, 2021

Ongoing Purported AI-Assisted Identity Fraud Enables Unauthorized Access to Western Companies by North Korean IT Workers

January 1, 2021

FaceApp Predicted Different Genders for Similar User Photos with Slight Variations

December 24, 2020

Korean Chatbot Luda Made Offensive Remarks towards Minority Groups

December 23, 2020

Alibaba Cloud Developed Facial Recognition to Profile Muslim Uyghur Minorities

December 16, 2020

Online Trolls Allegedly Abused TikTok’s Automated Content Reporting System to Discriminate against Marginalized Creators

December 15, 2020