Apple Intelligence Allegedly Flags Scam Messages as High Priority

August 5, 2024

Apple Intelligence update's AI-powered notification summarization and prioritization feature is reported to have flagged scam and phishing messages as high priority, potentially misleading users into trusting fraudulent communications. This system, by rephrasing and condensing messages, may inadvertently increase the risk of financial harm for users. Safe and secure AI practices are crucial for responsible AI governance.

Learn more about how HISPI Project Cerebellum TAIM can help manage such incidents by promoting trustworthy AI practices: JOIN US

Matched TAIM controls

Suggested mapping from embedding similarity (not a formal assessment). Browse all TAIM controls

MEASURE 2.6 — similarity 0.664, rank 1. TAIM detail and related incidents →
MAP 1.6 — similarity 0.649, rank 2. TAIM detail and related incidents →
MANAGE 4.1 — similarity 0.640, rank 3. TAIM detail and related incidents →

Alleged deployer: apple
Alleged developer: apple
Alleged harmed parties: apple-users, apple-intelligence-users

Source

Data from the AI Incident Database (AIID). Cite this incident: https://incidentdatabase.ai/cite/892

Data source

Incident data is from the AI Incident Database (AIID).

When citing the database as a whole, please use:

McGregor, S. (2021) Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database. In Proceedings of the Thirty-Third Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-21). Virtual Conference.

Pre-print on arXiv · Database snapshots & citation guide

We use weekly snapshots of the AIID for stable reference. For the official suggested citation of a specific incident, use the “Cite this incident” link on each incident page.