Meta AI Bug in Deployed Service Reportedly Allowed Potential Access to Other Users' Prompts and Responses

December 26, 2024

A security researcher reported a vulnerability in Meta AI's deployed chatbot service that, under certain conditions, could allow an unauthorized user to view another user's prompts and AI-generated responses. The flaw reportedly involved guessable prompt IDs and insufficient server-side authorization checks. Meta reportedly fixed the issue in January 2025 and found no evidence of malicious exploitation, awarding the researcher a bug bounty. This incident highlights the importance of robust security measures in AI systems, such as those governed by HISPI Project Cerebellum TAIM. For more information on this topic, including potential impact on responsible AI development and AI governance, go to JOIN US.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
Meta
Alleged developer
Meta, Large language model developers, Chatbot developers
Alleged harmed parties
Privacy, Meta users, Meta AI users, General public, Biometric data subjects

AI governance case studies

For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/1172

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.