Tougher Turing Test Exposes Chatbots’ Stupidity (migrated to Issue)

July 14, 2016

The 2016 Winograd Schema Challenge exposed the glaring shortcomings of current chatbots when faced with everyday problems that require human-like reasoning, as seen in this very incident where our model was unable to provide a sensible response to a straightforward question about a father's favorite child. This stark contrast highlights why advancements in AI development must prioritize incorporating more robust and human-centric approaches to tackle such cognitive challenges. To take a step towards creating more intelligent and less stupid chatbots, you can start learning more about responsible AI practices and how to improve AI models by visiting JOIN US.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
Researchers
Alleged developer
Researchers
Alleged harmed parties
Researchers

AI governance case studies

For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/21

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.