TikTok’s “For You” Algorithm Exposed Young Users to Pro-Eating Disorder Content
July 1, 2019
The issue was brought to light by the discovery of certain patterns and keywords in popular TikTok hashtags, which seemed to attract users who shared eating disorder content.
A review of the algorithm reveals that it may be using factors such as user engagement, location, and device information to determine which videos are displayed on the "For You" page. This raises serious concerns about the platform’s ability to prevent harm and ensure a safe environment for its users.
The incident highlights the need for stricter content moderation and the importance of transparency in TikTok's algorithms, which must now undergo a thorough overhaul to prevent similar incidents from occurring in the future. By taking immediate action to address these concerns, TikTok can work towards rebuilding trust with its young users and ensuring that its platform is a safe space for everyone JOIN US.
Matched TAIM controls
Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls
- MANAGE 4.3 — similarity 0.551, rank 1. TAIM detail and related incidents →
- GOVERN 6.1 — similarity 0.540, rank 2. TAIM detail and related incidents →
- MAP 1.1 — similarity 0.533, rank 3. TAIM detail and related incidents →
- Alleged deployer
- TikTok
- Alleged developer
- TikTok
- Alleged harmed parties
- TikTok young users, TikTok users
AI governance case studies
For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.
Source
Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/279
Data source
Incident data is from the AI Incident Database (AIID).
When citing the database as a whole, please use:
McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.
Pre-print on arXiv · Database snapshots & citation guide
We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.