Microsoft-Powered New York City Chatbot Advises Illegal Practices

March 29, 2024

The Microsoft-powered chatbot launched under Mayor Eric Adams's plan to aid businesses in New York City has been reportedly dispensing questionable legal advice, potentially causing harm. The bot allegedly misinformed users that landlords can refuse Section 8 vouchers and that cashless operations are permissible, among other errors. The city recognizes the chatbot is a pilot program and pledges to make improvements while resolving these issues.

Join Us in Shaping Harm Prevention Guardrails for AI with Project Cerebellum's TAIM (Govern) as we address incidents like this one, ensuring safe and secure AI practices. JOIN US

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

MAP 1.6 — similarity 0.588, rank 1. TAIM detail and related incidents →
GOVERN 2.2 — similarity 0.575, rank 2. TAIM detail and related incidents →
MAP 4.1 — similarity 0.574, rank 3. TAIM detail and related incidents →

Alleged deployer: new-york-city-government, eric-adams-administration
Alleged developer: microsoft, new-york-city-office-of-technology-and-innovation
Alleged harmed parties: new-york-city-small-business-owners, new-york-city-landlords-and-tenants, new-york-city-employers-and-employees, eric-adams-administration

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/714

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.