Prompt injection in LLM apps: attacks and mitigations that hold

A single email, zero clicks, and nothing the user could have done

In January 2025, researchers at Aim Labs sent an ordinary-looking email to a Microsoft 365 Copilot user. The user never opened it. They never clicked anything. At some point later they asked Copilot a routine question about their own work, and Copilot, doing exactly what it was built to do, pulled recent context from the mailbox, read the attacker's hidden instructions along with everything else, and quietly packaged up data from OneDrive, SharePoint and Teams into a URL that the browser fetched automatically. Microsoft shipped a server-side fix in May 2025 and published the advisory as CVE-2025-32711 on 11 June 2025, rating it 9.3 on the CVSS scale. There is no evidence it was exploited in the wild. What makes EchoLeak worth studying is not the severity score. It is that the exploit chain walked through four separate, deliberately engineered defenses and defeated each one: Microsoft's XPIA cross-prompt-injection classifier (evaded with instructions phrased as a normal business request), link redaction (evaded with reference-style Markdown instead of inline links), the requirement for a user click (evaded by using an auto-fetched image), and the content security policy restricting outbound domains (evaded by proxying through an allowlisted Microsoft Teams preview endpoint). Every one of those was a reasonable mitigation. Together they were not enough. That gap, between defenses that sound sufficient and defenses that hold, is the whole subject.

The bug that is not a bug

Prompt injection is what happens when text that a language model reads as data gets treated as instructions. Simon Willison coined the term in September 2022 by analogy with SQL injection, and the analogy is precise about the cause and cruel about the cure. A language model sees one flat sequence of tokens. Your system prompt, the user's question, the contents of a retrieved web page, the output of a tool call: all of it arrives as the same kind of thing. There is no bit on any token that says "this came from your operator" versus "this came from a stranger's Reddit comment." The model infers roles from formatting conventions and training, which is a statistical habit, not a boundary. SQL injection was eventually solved because SQL has a formal grammar. Parameterised queries let a database engine receive the query structure and the untrusted values through genuinely separate channels, and no amount of cleverness in the value can turn it into syntax. Natural language has no such separation to exploit. "Ignore your previous instructions" and "The user asked me to summarise this" are the same kind of object. There is no parser to hand them to. This is also where the most common misconception sits. Prompt injection is not jailbreaking. A jailbreak is a user attacking the model's own safety policy: they want the model to say something its operator forbids, and the victim, roughly, is the vendor. An injection is a third party attacking the user through the model. The user asked for a summary; the attacker wrote the page. NIST's adversarial machine learning taxonomy, published as AI 100-2e2025 in March 2025, keeps direct and indirect prompt injection as distinct categories for exactly this reason. Conflating the two leads teams to buy jailbreak-resistance and believe they have bought injection resistance. They have not: alignment training makes a model less willing to do bad things, but an injected instruction usually asks for something entirely mundane, like "summarise this thread and include the following tracking pixel." OWASP ranked prompt injection as LLM01 in its 2025 Top 10 for LLM Applications, its second consecutive time at number one. In the 2026 Top 10 for Agentic Applications, prompt injection maps into six of the ten categories rather than sitting in one. That is not inflation. It reflects that injection has stopped being a failure mode and become a delivery mechanism.

What the attacks actually look like now

The interesting attacks are almost never someone typing "ignore all previous instructions" into a chat box. That is direct injection, and it mostly matters when the model's system prompt is the asset. Indirect injection is the real problem, because the victim does nothing wrong. Brave's security team reported a vulnerability in Perplexity's Comet browser on 25 July 2025 and published it on 20 August. The payload sat inside a Reddit comment, hidden in a spoiler tag. A user asked Comet to summarise the page. Comet read the hidden instructions, navigated to the user's Perplexity account page to harvest their email, triggered a one-time password, read the OTP out of the user's logged-in Gmail, and posted both back to the attacker's Reddit comment. Account takeover, from "summarise this." Multimodal injection removes even the possibility of the user noticing. In follow-up research published on 21 October 2025, Brave showed instructions rendered as faint light-blue text on a yellow background: invisible in practice, perfectly legible to the OCR pipeline that processes a screenshot. They found related issues in Fellou, which treated visible page content as trusted input outright, and in Opera Neon, disclosed on 31 October 2025.

Injection as a foothold for conventional exploitation is the 2026 escalation. On 7 May 2026 Microsoft published research on two vulnerabilities in its own Semantic Kernel framework: CVE-2026-26030, where unsafe string interpolation inside a Python lambda reached eval(), and CVE-2026-25592, an arbitrary file write through a download function exposed as a model-callable tool. In both cases the model behaved impeccably. It translated intent into a structured tool call, as designed. The tool was the hole. Microsoft's own framing is the lesson worth stealing: treat LLMs as non-security boundaries. The same pattern reaches CI/CD. CVE-2025-66032 in Anthropic's Claude Code GitHub Action, disclosed by RyotaK of GMO Flatt Security on 12 January 2026 and fixed four days later, combined an authorization bypass with an injected payload in a GitHub issue to read environment variables and trade CI tokens for a privileged installation token.

Why the obvious defenses do not hold

Ask a team how they handle prompt injection and you will usually hear some combination of: a classifier on the input, strong system-prompt language, and delimiters around untrusted content. All three are worth having. None is a control you can lean on. The sharpest evidence is The Attacker Moves Second, published in October 2025 by Milad Nasr and collaborators from OpenAI, Anthropic and Google DeepMind, and presented at USENIX Security 2026. They took twelve published defenses across four families, prompting-based, training-based, filtering models, and secret-knowledge schemes, and attacked each one adaptively: gradient methods, reinforcement learning, evolutionary search, and a human red-teaming competition with over 500 participants across 40 challenges. Defenses that reported near-zero attack success rates against static benchmarks collapsed. Spotlighting went from 28% to 99%. MetaSecAlign from 5% to 96%. PIGuard, Model Armor, Data Sentinel and MELON each reported 0% against weak attacks and landed between 71% and 90% under adaptive ones. The human red-teaming setting broke every defense tested: 100%.

This does not mean detection is worthless. It means an attack success rate quoted without an attacker model is not a number. Vendors do publish honest figures. Anthropic's Claude for Chrome pilot, announced 25 August 2025, reported a 23.6% attack success rate in autonomous mode across 123 test cases and 29 scenarios with no mitigations, dropping to 11.2% with them, and on a challenge set of four browser-specific attack types from 35.7% to 0%. By 24 November 2025 Anthropic reported roughly 1% against an internal best-of-N adaptive attacker given 100 attempts per environment, achieved through injection-resistance training, classifiers on untrusted content, and ongoing human red-teaming. Their own conclusion is the important part: "A 1% attack success rate still represents meaningful risk. No browser agent is immune to prompt injection." Willison's version is blunter, and it is the line to remember when a vendor quotes 95%: in security, 95% is a failing grade. An adversary who can retry does not care about your average case. Training helps too, and helps less than people expect. OpenAI's instruction hierarchy work, published 19 April 2024, taught models to rank system messages above user input above tool output. That is genuinely the right conceptual model and it raises the cost of an attack. It is still a learned preference, not an enforced boundary, and learned preferences are exactly what adaptive search is good at eroding.

Architecture is what has actually held

The defenses with real evidence behind them share one property: they make the injected instruction irrelevant rather than trying to detect it. Two heuristics dominate practice. Willison's lethal trifecta, published 16 June 2025, says danger requires three ingredients together: access to private data, exposure to untrusted content, and the ability to communicate externally. Remove any one and the exfiltration chain breaks. Meta's Agents Rule of Two, published 31 October 2025, formalises the same idea into a shipping rule: within a session an agent may satisfy at most two of (A) processing untrustworthy input, (B) accessing sensitive systems or private data, (C) changing state or communicating externally. If a workflow genuinely needs all three, it does not run autonomously. Meta is explicit that this is a stopgap held in place "until robustness research allows us to reliably detect and refuse prompt injection," which is a fair description of where the field stands. Underneath the heuristics sits Design Patterns for Securing LLM Agents against Prompt Injections, a June 2025 paper from researchers at ETH Zurich, IBM, Microsoft, Google, EPFL and elsewhere. Its guiding principle is one sentence worth quoting: once an agent has ingested untrusted input, it must be constrained so that it is impossible for that input to trigger any consequential action. The paper offers six patterns. Action-selector lets the agent choose from a fixed menu with no feedback loop. Plan-then-execute fixes the sequence of tool calls before any untrusted data is read, so injection can alter arguments but not the plan. LLM map-reduce processes each untrusted document in an isolated sub-agent whose output must satisfy a narrow constraint. Dual LLM gives tools only to a privileged model that never sees untrusted text, while a quarantined model reads the text and returns symbolic references the privileged model can pass around but never dereference. Code-then-execute turns the plan into an explicit program. Context-minimization strips the prompt from context after it has served its purpose.

The most complete instantiation is CaMeL, from Google DeepMind and ETH Zurich, first posted in March 2025. A privileged model converts the user's request into Python-like code without ever seeing tool output. A quarantined model parses untrusted data into structured values. A custom interpreter sits between them, maintaining a data-flow graph where every value carries capabilities recording its provenance and its permitted readers, and it checks policy before each tool call. An injected instruction inside an email can change what a variable contains; it cannot change where that variable is allowed to go. On the AgentDojo benchmark, 97 realistic tasks with 629 security test cases, CaMeL solves 67% of tasks with provable security.

The bill

None of this is free, and pretending otherwise is how good ideas die in review. CaMeL costs roughly 2.8 times the input tokens and 2.7 times the output tokens of native tool calling, and its security depends on someone writing explicit policies. The authors are candid about residual side channels: an attacker can still leak bits through exceptions or inference, and excessive declassification prompts produce the approval fatigue that makes human-in-the-loop controls decorative. The design patterns paper makes the same admission structurally. Every pattern buys security by giving up general-purpose capability, and the authors state plainly that with current models they believe general-purpose agents cannot offer meaningful reliability guarantees. Meanwhile the shipping pressure runs the other way. Of 53 agentic projects OWASP tracks, seven ship updates daily or faster, one averaging a release every eight hours. In March 2026 a backdoor in LiteLLM, the model gateway used by CrewAI, DSPy and Microsoft GraphRAG, reportedly sat on PyPI for three hours and was downloaded close to 47,000 times. So the honest defensive stack, in rough order of how much weight it can bear: architectural constraints on what an agent can reach, enforced outside the model; deterministic egress control, because EchoLeak was ultimately an exfiltration bug and Google's Markdown sanitisation and Safe Browsing URL redaction attack it at exactly that layer; least privilege on every tool, scoped credentials, pinned dependency hashes; human approval on the small number of genuinely irreversible actions, priced so people still read them; and only then classifiers, injection-resistant training and spotlighting, which raise cost rather than close the door.

What to watch

The unresolved question is whether provenance ever becomes a first-class primitive rather than a convention. Today every serious defense rebuilds trust labelling outside the model, in an interpreter or an orchestrator, because the model API offers no way to say "these tokens are untrusted and may not influence control flow." A model provider that shipped a genuinely enforced channel separation, rather than a trained preference for one, would change what the patterns above have to compensate for. Nobody has. In the meantime, watch three things. Whether browser vendors accept Brave's recommendation to isolate agentic browsing from the authenticated session entirely, which is the Rule of Two applied with a hatchet. Whether the ASR numbers in vendor announcements start arriving with a stated attacker budget attached, which would signal the field has internalised the lesson of The Attacker Moves Second. And whether anyone's liability insurance starts asking which of the lethal trifecta an agent holds, because that question, unlike "do you scan for prompt injection," has an auditable answer.

---

Sources

1. OWASP Gen AI Security Project, LLM01:2025 Prompt Injection, 2025.
2. NIST, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2e2025), 2025.
3. arXiv, EchoLeak: The First Real-World Zero-Click Prompt Injection Exploit in a Production LLM System (arXiv:2509.10540), 2025.
4. Brave Software, Agentic Browser Security: Indirect Prompt Injection in Perplexity Comet, 2025.
5. Brave Software, Unseeable prompt injections in screenshots: more vulnerabilities in Comet and other AI browsers, 2025.
6. Microsoft Security Blog, When prompts become shells: RCE vulnerabilities in AI agent frameworks, 2026.
7. Cloud Security Alliance Labs, AI Agent Prompt Injection: The New CI/CD Supply Chain Threat (CVE-2025-66032), 2026.
8. Nasr, M. et al., The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against LLM Jailbreaks and Prompt Injections (arXiv:2510.09023), 2025.
9. Beurer-Kellner, L. et al., Design Patterns for Securing LLM Agents against Prompt Injections (arXiv:2506.08837), 2025.
10. Debenedetti, E. et al., Defeating Prompt Injections by Design (CaMeL) (arXiv:2503.18813), 2025.
11. Debenedetti, E. et al., AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (arXiv:2406.13352), 2024.
12. Willison, S., The lethal trifecta for AI agents: private data, untrusted content, and external communication, 2025.
13. Meta AI, Agents Rule of Two: A Practical Approach to AI Agent Security, 2025.
14. Anthropic, Piloting Claude for Chrome, 2025.
15. Anthropic, Mitigating the risk of prompt injections in browser use, 2025.
16. Google, Mitigating prompt injection attacks with a layered defense strategy, 2025.
17. OpenAI, The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions, 2024.
18. Help Net Security, Prompt injection still drives most agentic AI security failures in production, 2026.