WhatsApp has initiated a limited beta rollout of a feature called Scam Alert, which utilises an on-device machine learning model to identify potentially scam messages from non-contacts while maintaining end-to-end encryption.
Meta has provided a detailed technical overview to accompany this rollout and is expanding its Bug Bounty program to allow security researchers to test the system before a broader release.
How the Detection Model Works
When users enable Scam Alert, the app downloads a lightweight machine learning model directly to their device. This model classifies incoming messages from non-contacts by analyzing the conversational structure and linguistic patterns typically associated with known scams.
Importantly, no message content leaves the device for classification, and nothing is reported to WhatsApp, Meta, or any third party unless the user takes explicit action.
If a message is flagged, a warning appears only in the user’s chat view, allowing them to block, report, continue, or mark the chat as trusted, which suppresses future warnings for that conversation.
Privacy-Preserving Telemetry
| Component | Function | Privacy Safeguard |
|---|---|---|
| On-device inference | Classifies messages using linguistic and structural signals | Model and message data never leave the device |
| Confidential federated analytics | Measures whether the feature is triggering correctly | Processed inside Trusted Execution Environments (TEEs) |
| Warning counts | Tracks how often the model surfaces alerts | Aggregated, differentially private, no per-user data |
| User action counts | Tracks trust/block/report decisions | k-anonymity thresholds and DP noise applied before reaching Meta |
| OHTTP relay | Routes telemetry and model requests | Strips requester IP; authenticated via anonymous credentials |
| Model delivery | Distributes updated ML models | Published on a third-party append-only transparency ledger before deployment |
A significant security concern with any server-delivered model is targeted delivery, where an attacker or insider could push a manipulated model to a specific individual. Meta addresses this issue by publishing every model version, identified by its SHA-256 hash, to a public transparency ledger before deployment.
This process is signed by a third party (Cloudflare), not Meta itself. Download requests are routed through OHTTP relays with anonymous credentials, ensuring the server cannot identify which user is requesting a model.
Additionally, assignment to experimental groups for new model variants occurs entirely on-device using locally generated randomness.
The measurement pipeline builds on Meta’s peer-reviewed PAPAYA Federated Analytics Stack, utilizing confidential virtual machines (CVMs) as the Trusted Execution Environment.
Data flows through job selection, local transformation into anonymous counts, RA-TLS attestation with the orchestrator TEE, and secure aggregation; individual metrics are discarded immediately after being merged into running histograms.
The threat model explicitly considers third-party and supply-chain vendors, malicious insiders, and external attackers. Defense-in-depth measures include encrypted DRAM, CVM hardening, and non-targetable relay routing.
Users can review their Scam Alert activity by navigating to Account > Request Info > Scam Alert Activity. This section logs which messages were scanned, the classification outcomes, and the model version used.
Before the beta launch, Meta engaged external researchers through its Bug Bounty program to review the privacy architecture and model integrity.
The company plans to expand this scope to include the confidential federated analytics pipeline and model weights directly. Independent sources noted that the feature was initially spotted in the WhatsApp Android beta version 2.26.22.2 before this formal technical disclosure.
Industry outlets, including SecurityWeek, PCMag, and Social Media Today, have highlighted Scam Alert as a significant case study in balancing on-device AI detection with guarantees of end-to-end encryption, emphasizing the commitment to verifiability over the detection feature itself.
Stop new phishing & malware before they compromise your business. Integrate live intel from 15K SOCs around the world
