Proctoring Algorithm in Online California Bar Exam Flagged an Unusually High Number of Alleged Cheaters

December 4, 2020

The proctoring algorithm used in a California bar exam cited a third of thousands of applicants as cheaters, resulting in allegations where exam takers were instructed to prove otherwise without seeing their incriminating video evidence. This raises concerns about the reliability and fairness of online assessments, particularly those involving complex subjects like law. As such, it is imperative that we prioritize responsible AI development and governance to ensure safe and secure AI practices. For more information on this incident and its implications for Project Cerebellum TAIM, visit JOIN US.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
California Bar’s Committee of Bar Examiners
Alleged developer
ExamSoft
Alleged harmed parties
flagged California bar exam takers, California bar exam takers

AI governance case studies

For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/131

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.