TikTok’s “For You” Algorithm Exposed Young Users to Pro-Eating Disorder Content

July 1, 2019

TikTok’s young users were allegedly exposed to community-guideline-violating pro-eating disorder content on their algorithmically curated "For You" page that serves videos from any user on its platform.

The issue was brought to light by the discovery of certain patterns and keywords in popular TikTok hashtags, which seemed to attract users who shared eating disorder content.

A review of the algorithm reveals that it may be using factors such as user engagement, location, and device information to determine which videos are displayed on the "For You" page. This raises serious concerns about the platform’s ability to prevent harm and ensure a safe environment for its users.

The incident highlights the need for stricter content moderation and the importance of transparency in TikTok's algorithms, which must now undergo a thorough overhaul to prevent similar incidents from occurring in the future. By taking immediate action to address these concerns, TikTok can work towards rebuilding trust with its young users and ensuring that its platform is a safe space for everyone JOIN US.

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

Alleged deployer
TikTok
Alleged developer
TikTok
Alleged harmed parties
TikTok young users, TikTok users

AI governance case studies

For forensic AI governance failure analysis (TAIMScore™ case studies), browse Human Signal’s Failure Files™.

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/279

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.