Optum Algorithmic Health Risk Scores Reportedly Underestimated Black Patients' Needs

October 24, 2019

The discovery highlights a critical oversight in AI-driven healthcare systems, where algorithms may not accurately capture the unique experiences and needs of historically marginalized populations. As researchers emphasize the need for more diverse and inclusive data sets, this incident underscores the imperative of addressing bias in healthcare technology to ensure equitable access to care for all patients. We encourage you to JOIN US.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
unnamed large academic hospital
Alleged developer
Optum
Alleged harmed parties
Black patients

AI governance case studies

For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/124

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.