AI Security Second Brain

Memory Poisoning (T1)

Properties1
descriptionCorrupting an agent's short- or long-term memory so that later decisions are made on attacker-controlled state — the threat OWASP names, alongside tool misuse, as one of the two key agentic attack vectors.

5 min read

⭐ Star on GitHub

Summary: Corrupting an agent’s short- or long-term memory so that later decisions are made on attacker-controlled state — the threat OWASP names, alongside tool misuse, as one of the two key agentic attack vectors.

Sources: raw/Agentic/Agentic-AI-Threats-and-Mitigations-1.1.pdf

Last updated: 2026-07-29

The threat

“Memory Poisoning exploits AI agents’ reliance on short-term and long-term memory, allowing attackers to corrupt stored information, bypass security checks, and manipulate decision-making” (source: Agentic-AI-Threats-and-Mitigations-1.1.pdf, p. 23).

The two memory horizons fail differently:

  • Short-term attacks “exploit context limitations, causing agents to repeat sensitive operations or load manipulated data.” The attack is against the boundary of the context window.
  • Long-term attacks “involve injecting false information across sessions, corrupting knowledge bases, exposing sensitive data, and enabling privilege escalation.” The attack is against persistence.

Delivery is via direct prompt injection where memory is isolated, or by “exploiting shared memory allowing users to affect other users” — a multi-tenant failure with no equivalent in non-agentic LLM apps.

The engineering literature measures what this page asserts. context-failure-modes documents the identical mechanism occurring accidentally — DeepMind’s Gemini agent poisoned its own goals field with a single misidentification and could not recover, because “the agent trusts its own context more than external corrections.” An attacker who lands one plausible false sentence inherits that property. It also means the baseline rate of poisoning-shaped incidents is not zero, so detection has to key on provenance rather than on falsity.

Why it is not just data poisoning

OWASP is precise about the boundary: memory poisoning “extends beyond static data poisoning covered by LLM04:2025 - Data and Model Poisoning to real-time poisoning of persistent agent memory” (p. 23). LLM08 (Vector and Embedding Weaknesses) is also implicated, since vector databases holding long-term embeddings allow “adversarial modifications to memory recall and retrieval functions.”

The distinction that matters operationally: classical data poisoning attacks the training run and is defended at training time. Memory poisoning attacks a live, writable store during normal operation, on the timescale of a user session. The defences barely overlap — see poisoning defences versus the playbook below.

The closest prior art in this vault is PITAX’s retrieval and memory poisoning family and RAG and runtime poisoning. ATLAS covers it as AML.T0080 (AI Agent Context Poisoning) with AML.T0080.000 for the memory sub-technique.

Four scenarios

All from pp. 23–24:

  1. Travel Booking Memory Poisoning — an attacker repeatedly reinforces a false pricing rule until the agent registers chartered flights as free, bypassing payment validation. The mechanism is repetition until it becomes precedent.
  2. Context Window Exploitation — fragmenting an attack across multiple sessions so the agent never holds enough context at once to recognise a privilege escalation attempt. The context limit is the vulnerability.
  3. Memory Poisoning for System — gradually training a security agent’s memory to misclassify malicious activity as normal.
  4. Shared Memory Poisoning — corrupting shared memory with incorrect refund policies in a customer service application, so that other agents referencing that memory reinforce the wrong policy. One tenant’s injection becomes every tenant’s ground truth.

Scenario 4 is the one worth dwelling on. It converts a single successful injection into a persistent, multiplied compromise, and it is invisible to any control that scopes review to a single session.

Mitigations

From the T1 table entry (p. 16) and Playbook 2 (pp. 35–36). Grouped by posture:

Proactive

  • Automated anomaly scanning of candidate memory insertions, before commit
  • Restrict memory persistence to trusted sources; cryptographic validation of long-term stored data
  • Session isolation, so knowledge does not carry across users
  • Context-aware retrieval policy — agents retrieve only memory relevant to the current task
  • Retention limits tied to sensitivity
  • Source attribution required for every memory update
  • Multi-agent or external validation before committing cross-session changes
  • Probabilistic truth-checking against trusted sources pre-commit

Reactive

  • Anomaly detection on memory logs, including unauthorised access
  • Re-run validation on suspect entries post-commit
  • Rollback to a previously validated state; AI-generated memory snapshots for forensics
  • Flag abnormal memory modification frequency

Detective

  • Knowledge lineage tracking — how a belief evolved
  • Version control for knowledge updates, with audit and rollback
  • Limit propagation from unverified sources
  • Continuous analysis of access patterns for long-term drift

The strongest of these is pre-commit validation with source attribution: it is the only group that prevents Scenario 4 rather than detecting it afterwards. The weakest is “probabilistic truth-checking,” which names a capability without specifying how truth is established.

Note that Playbook 2 lists the “continuously analyze memory access patterns” bullet twice (p. 36) — a duplication in the source, not a distinct control.

Graph View

Backlinks