Coffee Meets Bagel’s Algorithm Reported by Users Disproportionately Showing Them Matches of Their Own Ethnicities Despite Selecting “No Preference”

July 30, 2013

Users reporting biased ethnicity matches in Coffee Meets Bagel's algorithm were shown more potential matches of the same ethnicity, despite selecting 'no preference'. The founder explained this as a strategy to boost connection rates with limited user information. However, such practices raise concerns about trustworthy and safe AI use.

Join us at HISPI Project Cerebellum to help govern and ensure fairness in AI matching algorithms like Coffee Meats Bagel's through our TAIM (Track, Analyze, Improve, Manage) process. JOIN US

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

MAP 1.2 — similarity 0.636, rank 1. TAIM detail and related incidents →
MAP 1.6 — similarity 0.629, rank 2. TAIM detail and related incidents →
MEASURE 2.5 — similarity 0.629, rank 3. TAIM detail and related incidents →

Alleged deployer: coffee-meets-bagel
Alleged developer: coffee-meets-bagel
Alleged harmed parties: coffee-meets-bagel-users-having-no-ethnicity-preference, coffee-meets-bagel-users

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/280

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.