Summary: Four documented attacks on production enterprise AI — AgentFlayer, Microsoft 365 Copilot, PoisonedRAG, Nightshade. The threat most organisations actually face, because you don’t train models but you do feed documents into AI workspaces.
Sources: raw/Data-poisoning/Data Poisoning and Generative AI_ What do companies need to know_.md (Trilateral Research, 2025-10-29)
Last updated: 2026-07-28
Why this is the practical threat
“Whilst you cannot control your vendor’s training pipeline, you do feed data into the AI workspace systems you use (think of SharePoint, Google Drive, Confluence, Slack, and email). An attacker who can introduce malicious content into these sources can manipulate AI outputs without accessing the underlying model or getting the user to take an action” (source: Trilateral Research).
That last clause is the distinction from phishing. There is no click to avoid.
The stated failure mode: “degraded performance, biased results, data being exposed to unauthorised parties, and answers that confidently cite non-existent information. These often emerge gradually and may be dismissed, until the business impact becomes large enough for the issue to be investigated.”
AgentFlayer — poisoned Google Drive file (August 2025)
Zenity researchers showed “a single hidden 300-word payload inside a shared Google Drive file could silently compromise ChatGPT via Connectors and exfiltrate secrets with no further user action required.”
Impact: “Any organisation connecting AI assistants to SaaS data stores (Drive, GitHub, calendars, internal wikis) can be compromised via a single malicious file entering shared spaces. OpenAI shipped mitigations following disclosure, but the underlying architectural vulnerability persists: AI agents with broad access permissions represent high-value attack surfaces.”
Microsoft 365 Copilot — ASCII smuggling (August 2024)
Malicious content in emails or documents instructing Copilot “to auto-invoke tools, retrieve additional emails and SharePoint files, and exfiltrate data through ‘ASCII smuggling’ techniques.”
The delivery detail: “the attacker sends an email that does not need to be opened. Instead, the attack occurs when Copilot processes it in the background, hidden instructions trigger data exfiltration automatically.”
Generalisation offered: “any AI assistant that can read your emails, access your file storage, and take actions on your behalf (such as Slack AI, Google Workspace AI, or custom RAG systems connected to your internal databases) faces the same fundamental risk.”
PoisonedRAG — knowledge corruption (USENIX Security ‘25)
“Inserting a small number of malicious texts into a RAG knowledge base can reliably influence outputs through ‘knowledge corruption’ without triggering system failures or obvious alerts.”
The two worked examples are worth keeping because they are boring, which is the point:
- “A pricing model that systematically underquotes after someone plants a document suggesting lower margins represent competitive best practice”
- “A contract review tool that overlooks liability clauses because a poisoned legal memo taught it those terms are standard boilerplate”
“The danger is not a catastrophic failure, but confidently incorrect answers that compound over time and influence strategic decisions.”
Nightshade — image model poisoning (January 2024 onward)
University of Chicago tooling letting creators “subtly alter images so models trained on them learn incorrect associations” — teaching a model that “‘handbag’ should be classified as a ‘toaster’.”
Risk to organisations “training or retraining vision models on scraped or user-submitted content”: visual search returning wrong products, medical imaging tools learning wrong associations, and “quality degradation that proves difficult to diagnose because the training data appears legitimate.”
Mitigations
Trilateral’s four, which are access-control problems rather than ML problems:
- Control who can add content to your AI systems — the ingestion surface is the attack surface.
- Limit what your AI assistants can do automatically — constrain tool invocation and autonomous action.
- Track what your AI is reading and using — provenance and telemetry at retrieval time.
- Embed human review where needed — at decision points with material consequences.
Point 2 is the one the vault’s other sources converge on. Compare PITAX T69, where a ReAct agent “rarely re-evaluate[s] policy between steps,” and ATLAS, where AI Agent Tool Invocation is the most-mitigated technique in the matrix at 11 mitigations.
How this connects across the vault
These four attacks are the real-world instances of techniques catalogued elsewhere:
| Attack | PITAX | ATLAS |
|---|---|---|
| AgentFlayer | indirect injection, tool abuse | AML.T0051.001 Indirect; AML.T0053 Tool Invocation |
| M365 Copilot | encoding + tool invocation | AML.CS0037-adjacent; Exfiltration tactic |
| PoisonedRAG | T64 retrieval ranking manipulation | Resource Development |
| Nightshade | training-time poisoning | Resource Development |
ATLAS carries a directly comparable case, AML.CS0037 Data Exfiltration via Agent Tools in Copilot Studio (Zenity, June 2025) — see ATLAS case studies.