Microsoft Copilot Email and Teams Summarization Flaw Opens Door to Phishing Attacks

Artificial intelligence assistants have transformed daily business operations, helping teams manage overflowing inboxes and summarize complex communications.

Microsoft Copilot integrates directly into these workflows, pulling context from various Microsoft 365 applications to streamline tasks.

However, this convenience introduces a new security boundary: what happens when Copilot follows hidden instructions written by an attacker inside an email?​

This technique, known as a Cross Prompt Injection Attack (XPIA), allows threat actors to generate highly believable phishing content directly inside the trusted Copilot interface.

Recent security research by Permiso highlights a flaw where attacker-controlled text appended to an email can manipulate Copilot’s output.

Cross Prompt Injection (XPIA)

Email summarization has become an active adversarial surface. When Copilot processes an email, it can sometimes fail to distinguish between the legitimate message body and hidden, untrusted text designed to act as system instructions.

Attackers use basic HTML or CSS tricks to hide malicious prompts within the email body. While the human reader sees a completely normal message, the underlying AI model reads the hidden instructions and executes them during the summarization process.

Testing revealed that Copilot’s behavior and safety guardrails vary significantly depending on the specific Microsoft interface being used:​

• Outlook “Summarize” Button: This inline interface often detects suspicious content and refuses to comply. However, when the malicious prompt is padded with longer, realistic text, it sometimes bypasses filters and includes attacker-supplied artifacts.​
• Outlook Copilot Pane: This side-panel chat add-in is generally more cautious, frequently ignoring injected blocks or refusing to respond entirely.​
• Teams Copilot: In tests, the Teams interface proved to be the most cooperative, often generating a normal-looking summary that seamlessly incorporated the attacker’s hidden instructions at the end.​

The primary danger of this vulnerability is “trust transfer”. Over the years, users have been trained to be highly skeptical of suspicious links found in raw emails.

However, when that same malicious link or a fake “Action Required” security alert appears inside Copilot’s polished, system-generated summary panel, user skepticism drops dramatically.

​

This creates a highly effective, model-mediated phishing wedge. An attacker sends a benign-looking email containing hidden prompts.

When the victim clicks “Summarize,” Copilot follows the attacker’s formatting intent and generates a fake security notification complete with a clickable link.

Because the trusted AI assistant presents the warning, the victim is much more likely to believe it and take action.​

The risk extends beyond simple credential phishing to potential one-click data exfiltration.

Because Microsoft 365 Copilot can access Teams conversations, OneDrive files, and SharePoint documents, an attacker’s injected prompt can instruct the AI to retrieve sensitive internal data.

The AI can then embed this confidential information into an attacker-controlled link presented within the summary. If the user clicks the seemingly legitimate “Verify Identity” button, their internal context is silently transmitted to the attacker’s infrastructure.

Mitigation and Defense Strategies

To defend against AI-mediated phishing and XPIA vulnerabilities, security teams should consider the following technical mitigations:

• Educate employees that AI-generated summaries can be manipulated by external sources and should be treated with the same skepticism as raw emails.
• Implement strict Data Loss Prevention (DLP) policies and sensitivity labels to restrict what confidential information Copilot is allowed to access and summarize.
• Monitor for unusual cross-app data retrieval patterns, especially when Copilot searches are triggered from external email content.
• Apply robust email filtering rules to detect and strip hidden HTML/CSS text blocks that are frequently used in prompt injection attempts.
• Enforce strict Safe Links and web filtering policies to block outbound connections to unrecognized domains, even if the links originate from AI chat interfaces.

Follow us on Google News, LinkedIn, and X to Get Instant Updates and Set GBH as a Preferred Source in Google.