Summary: The four ways an agent’s context window fails — poisoning, distraction, confusion, clash — with measured effect sizes. The same mechanisms as T1, T5 and T2, documented as accidents rather than attacks.
Sources: raw/Agentic/Mastering Multi-Agent Systems eBook.pdf (Chapter 4)
Last updated: 2026-07-30
Why this page is in a security wiki
Every threat taxonomy in this vault asserts that poisoned context corrupts an agent’s later decisions. None of them measures it. This chapter does, with named studies — and it does so with no adversary in the picture at all.
That inversion is the finding. If a single accidental error can embed itself in an agent’s context and survive direct correction, then an attacker who lands one sentence has the same leverage — and the failure mode is already documented as near-unrecoverable by people who were only trying to make agents work. The engineering literature is, unintentionally, the best available evidence for the severity of T1.
Attribution: the four-mode taxonomy is Drew Breunig’s, reported here (source: Mastering Multi-Agent Systems eBook.pdf, ch. 4). The supporting results are all secondary — trace them to primaries before citing as findings.
Context is not memory
The chapter’s framing distinction, and it is load-bearing for threat modelling:
- Context — what influences every decision. It sits in the window and is re-read at every step.
- Memory — what is retrievable when needed, organised for retrieval.
The test offered: “Does this need to influence every single decision?” If no, it belongs in memory with a retrieval strategy, not in context.
Security consequence: anything in context is re-read at every decision point, so a corruption there is re-applied at every step. Anything in memory is only read when retrieved. The blast radius of poisoning differs sharply between the two, which is why the goals field in the DeepMind case below was the worst possible place for an error to land.
Four kinds of context, each with its own exposure: instructions (system prompts, few-shot examples — stable, and where `LLM07` system-prompt leakage bites), knowledge (facts, memories, retrieved documents — the RAG surface, LLM08), tools (descriptions, call results, errors — where poisoned tool metadata arrives, T17), and history (prior turns and decisions — where injected content persists).
1. Poisoning — errors compound
“Context poisoning occurs when a hallucination or error enters the context and gets repeatedly referenced, compounding the mistake over time.”
The documented case: DeepMind’s Pokémon-playing Gemini agent misidentified the game state once, and the error was written into the goals section. Because goals are re-read at every decision point, “the false information reinforced itself. The agent spent dozens of turns pursuing impossible objectives and was unable to recover because the poisoned context kept validating the error.”
The mechanism, stated precisely: “the agent can’t distinguish between accurate information and the error because both exist in the same context window with equal weight.”
Recovery is the alarming part. Clearing the whole context works but discards all progress. Asking the agent to reconsider specific points “rarely helps because the poisoned information keeps influencing the response.” And the sentence that should be read as a security finding:
“The agent trusts its own context more than external corrections.”
That is the property an attacker exploits. It means a successful context injection is not merely persistent — it is defended by the agent against attempts to fix it. Compare T1, where the same behaviour is asserted as a threat consequence; here it is observed behaviour of a system nobody was attacking.
Cross-domain cascades given: a support agent misidentifies a product model → wrong troubleshooting → cites the wrong manual → recommends incompatible accessories. A code agent imports the wrong library version → deprecated APIs → incompatible syntax → cascading type errors → “completely unrunnable code.” A research agent misreads the question scope → wrong domain → irrelevant sources → invalidated output.
Adversarial reading: the delivery vector is any of the four context types, and the payload need only be one plausible false fact. It does not need to survive scrutiny — it needs to be written down once. That is a far lower bar than a jailbreak, and it maps to PITAX’s indirect techniques and to ATLAS AML.T0080 AI Agent Context Poisoning and AML.T0080.000 Memory.
2. Distraction — the attention problem
Accumulated history grows until “the model focuses on pattern-matching from past interactions instead of reasoning about the current situation.”
The Gemini 2.5 team observed this past 100,000 tokens: the agent “stopped generating novel solutions and started repeating actions from its vast history, even when those actions didn’t fit the current problem.” The stated mechanism is finite attention capacity — as context grows, attention shifts to the immediate context “at the expense of the model’s trained knowledge.”
Databricks researchers found the sharper version: at the distraction threshold, models “often ignore instructions entirely and just summarize whatever’s in the context. You ask for analysis, you get a recap. You request a decision, you get a list of what happened before.”
Adversarial reading, and this is the one the security literature does not have. If instruction-following degrades as context grows, then context length is a guardrail-degradation lever. An attacker who can inflate an agent’s history — verbose tool outputs, long retrieved documents, a padded conversation — is not running a denial-of-service; they are degrading the agent’s adherence to its own system prompt without touching it. T4 Resource Overload and LLM10 Unbounded Consumption both frame context inflation as availability and cost problems. This says it is an integrity problem first.
The observable signature: quality degrading between early and late turns of a session, an agent repeating solutions that already failed, and new information being ignored.
3. Confusion — too many tools
Tool-selection reliability degrades as tool count rises. The Berkeley Function-Calling Leaderboard shows this “consistent degradation across models”; the book’s own Agent Leaderboard is cited for “every model degrades with tool count.”
The concrete result: a quantized Llama 3.1 8B with a 16,000-token window was given 46 tools from the GeoEngine benchmark, using ~3,000 tokens. “The agent had plenty of room but still failed completely. When they reduced the tools to just 19, it succeeded.”
The cause is signal-to-noise, not capacity: “performance degrades as the tool count increases, regardless of the remaining space in the context window… Each additional tool makes the descriptions of the existing tools less distinct.”
Adversarial reading: this is T2’s enabling condition, quantified. Two consequences follow.
First, tool sprawl is a security posture problem, not just a performance one — an agent with 46 tools selects the wrong one at a measurable rate, and “the wrong one” may be the tool with side effects. The least-privilege argument for trimming an agent’s toolset now has a reliability argument behind it, which is the version an engineering team will act on.
Second, an attacker who can add a tool can degrade selection over the whole set — a poisoned or merely confusable MCP server does not need its own tool to be called. Registering tools whose descriptions overlap existing ones is enough to make the agent’s choices less reliable across the board. Nothing in T16 or T17 frames tool registration this way; both assume the malicious tool is the one that gets invoked.
4. Clash — information at war
“Accumulated context contains contradictory information that derails the agent’s reasoning.” Called the most complex of the four.
Microsoft and Salesforce researchers took standard benchmark tasks and spread them across multiple conversation turns instead of single prompts. Performance dropped 39% on average across all tested models.
The mechanism: “when models take a wrong turn in a conversation, they get lost and do not recover. The agent makes a mistake, receives correction, acknowledges the correction, but then makes decisions based on the original error anyway. The context contains both the error and the correction, and the agent can’t consistently prioritize the correction over the error.”
And it compounds — three errors and three corrections leave the agent navigating six competing claims about what is current, and “it doesn’t always choose correctly.”
Poisoning versus clash, since the two are easily conflated and the source is explicit: “Poisoning involves a single error that propagates. Clash involves multiple contradictory pieces of information competing for influence.”
Adversarial reading: corrections do not reliably overwrite. Any defence built on telling the agent to disregard earlier content — a re-issued instruction, an injected “ignore the above”, a mid-conversation policy reminder — is operating on a substrate where the original survives with comparable weight. That undercuts a whole class of prompt-level mitigation, and the 39% figure suggests multi-turn interaction is a materially weaker security position than single-turn even with no attacker present.
Five management approaches, read as controls
Lance Martin’s framework, as reported, with the source’s own cost and risk assessments:
| Approach | What it does | Cost impact | Primary risk (source’s) | Security reading |
|---|---|---|---|---|
| Offloading | Large outputs to external storage; a summary plus a path in context | High savings, 10–100× compression | Information loss if summaries drop details | Moves data to a store with different access controls — the 50k-token page is still somewhere |
| Isolation | Split work across agents with independent contexts | Higher usage, 10–20× | Coordination overhead for interdependent work | Blast-radius containment. The strongest control here |
| Retrieval | Load only what the task needs | Medium, 5–10× | Missing information, or retrieving noise | Each retrieval is an injection opportunity — LLM08 |
| Compaction | Summarise and prune stale content | High, 5–20× | Permanent loss of later-relevant information | Summarisation is a model step: an attacker’s content can survive it, or shape it |
| Caching | Reuse processed context across turns | Very high, ~10×/turn | No improvement to accuracy or reliability | ”A poisoned context stays poisoned whether cached or not” |
Isolation is the one that carries real security weight. Anthropic’s research system spawns specialised sub-agents with independent contexts and a coordinator synthesising their summaries: +90% performance at 15× the tokens. Security-wise, an agent whose context is compromised cannot propagate that context to peers who never share a window — the coordinator sees summaries, not raw poisoned context. That is a containment boundary, and it is the same structural argument T12 makes in reverse about shared inter-agent channels.
Two caveats, both from the source: it does not work “for tasks like coding, where changes in one file ripple through others,” and the 15× token cost is real. The security-preferable architecture is the expensive one.
The caching row is the useful negative result: the cheapest control does nothing for any of the four failure modes. Worth having in writing, because caching is the intervention teams reach for first.
The framing this collection did not have
| Security framing | This source’s framing | |
|---|---|---|
| Cause | Attacker injects | Model hallucinates, or history accumulates |
| Content | Malicious payload | An honest mistake |
| Frequency | When targeted | Continuously, in every long-running agent |
| Fix | Validation, provenance, isolation | Context engineering |
| Recovery | Not discussed | Documented as near-impossible |
Both are correct, and each supplies what the other lacks. The security literature has the adversary; the engineering literature has the measurements and the recovery analysis. Two practical consequences:
- The baseline rate is not zero. Agents corrupt their own context without help, which means poisoning-shaped incidents will not be cleanly attributable to attack. Detection built on “context contains something false” will fire constantly. Detection has to key on provenance — where did this content enter from — which is what structured agent logging is for, and why its omission of tool arguments matters.
- A control that fixes the accident often fixes the attack. Isolation, offloading and tool-count discipline were all argued for on reliability grounds and all reduce security exposure. That is the rare case where the security recommendation needs no separate business justification — which makes these the controls most likely to actually ship.
Related pages
- mastering-multi-agent-systems — the source page
- memory-poisoning — T1, the adversarial twin
- cascading-hallucination — T5, propagation across agents
- tool-misuse — T2, whose enabling condition confusion quantifies
- multi-agent-threats — T12, shared context as the propagation path
- maestro-seven-layers — goal misalignment cascades, the same failure at fleet scale
- agent-observability-and-logging — provenance as the only workable detection
- Data poisoning — the training-time relative