Summary: The six defensive layers of an AI system — data, weights, model behaviour, application, agent, organisation — which collection in this vault covers each, and which layers are genuinely thin.
Sources: synthesis across raw/MITRE-ATLAS/, raw/Laws/, raw/Agentic/, raw/Supply-chain-attacks/, raw/Data-poisoning/, raw/AI-incident-response/, raw/OWASP-Top-10-for-LLMs/
Last updated: 2026-07-29
No source in this vault presents a complete defensive stack. Each covers a layer or two and defers the rest. This page assembles the stack and marks where the corpus is strong, thin, or silent.
The six layers
| Layer | Asset | Primary source here | Depth |
|---|---|---|---|
| 1. Data | Training corpora, labels, RAG stores | Poisoning defences | Strong |
| 2. Weights | Unreleased model parameters | CoP Appendix 4 | Strong |
| 3. Model behaviour | What the model will say or refuse | ATLAS (guardrails, alignment, guidelines) | Moderate |
| 4. Application | Code that calls the model | Securing LLM applications | Moderate |
| 5. Agent | Systems that act | ASI playbooks | Moderate, mostly unimplemented |
| 6. Organisation | Process, people, procurement, IR | AI incident response, provenance and assurance | Strong |
Cutting across all six: provenance (layers 1, 2, 4) and telemetry (layers 3–6). See telemetry-and-logging.
Layer 1 — Data
The defensive position is settled and unusually clear: prevention over detection, because detection does not work at the sample counts involved. CMU’s argument is that “probabilistic methods to retroactively check whether data was tampered with are becoming less effective,” recovery via unlearning requires already knowing what was poisoned, and retraining is “extremely expensive” (source: CMU chain-of-custody article, via poisoning defences).
Controls: cryptographic chain of custody over the pipeline, signing the pairing of data and label rather than the data alone, least privilege on training data, source diversity, ensemble disagreement monitoring (MEDLEY), adversarial training.
Where it fails: the asymmetry. “Attackers need to compromise only a few hundred samples, while defenders must secure all data entry points.” See where-defenses-fail.
Layer 2 — Weights
The most prescriptive control set in the corpus, and the only one with hard numbers: 256-bit encryption, TPM key storage, attested trusted execution environments for parameters in use, a secure registry of every device holding a copy, physical inspection of data centres (source: Code of Practice Appendix 4.2).
The rationale for this being the deepest layer is sound — once weights leak, abliteration removes safety behaviour entirely and no model-level control survives. Weight confidentiality is the precondition for every control at layer 3.
Where it fails: Appendix 4 protects outbound weights and never verifies inbound artifacts. No dataset or model signing on the way in. Palo Alto’s recommendation of “cryptographic signing of models and datasets” has no Appendix 4 counterpart.
Layer 3 — Model behaviour
ATLAS carries the technique-level material: Generative AI Guardrails, Generative AI Guidelines, Generative AI Model Alignment, each with per-technique applicability text. For AML.T0051 prompt injection alone there are six mapped mitigations.
Where it fails: this is the layer the attack collections spend the most effort defeating, and largely succeed. Guardrails and alignment are probabilistic controls against an adversary who gets unlimited attempts — see automated attack generation. The honest position is the one from StackHawk: layered defence raises cost, it does not close the class.
Layer 4 — Application
Code that calls models. Three defences, in dependency order:
- Context isolation / spotlighting — structurally marking trusted vs untrusted context. Explicitly modest: “This doesn’t guarantee safety… but it significantly raises the bar.”
- Output validation — treat model output as untrusted input. This is
LLM05implemented. The sophisticated form is Google’s “User Alignment Critic,” a second model isolated from the potentially poisoned context. Isolation is the load-bearing property; a critic sharing the context inherits the compromise. - Least privilege for the model — explicit tool allow-lists, human approval for sensitive operations.
Plus AI gateways as a security control plane, and token-aware rate limiting — request-count limits are close to meaningless when one request can cost thousands of times another.
Where it fails: spotlighting’s own mechanism is a documented attack surface — delimiter isolation is exactly what T07 End Sequences and T48 Special-Token Injection attack.
Layer 5 — Agent
The six ASI playbooks, covering T1–T17. Highest-value implementable controls: execution sandboxes with destroy-after-use, JIT tool access, session isolation for memory, mutual agent-to-agent authentication, topology segmentation, and limiting an agent’s ability to render links.
Where it fails: two structural problems. First, roughly a third of the prescribed controls name capabilities that do not reliably exist — trust scoring, truthfulness verification, cryptographic attestation of agent integrity (which the source concedes does not exist for LLMs). Second, and worse, protocol-layer invocation bypasses the whole layer: MCP and A2A tool calls “may bypass user interfaces, consent flows, or direct human oversight,” and a stolen persistent agent identity bypasses “the agent’s conversational interface and its guardrails.”
Layer 6 — Organisation
The best-covered layer, and the least glamorous. Provenance and SBOM practice, procurement as a security control, staged containment, reporting clocks, red-teaming, bug bounties, personnel vetting, responder wellbeing.
Two findings worth carrying:
- Procurement already functions as a security control whether or not it is designed as one — 28% of organisations in the Plexal/Oxford study had changed vendors over trust or compliance issues.
- Containment is architecture-specific and legally constrained. AI Act Art 73(6) forbids investigation that alters the system in ways affecting subsequent evaluation before notifying authorities. Rolling back a model, purging memory and rebuilding a vector store are all such alterations. That sequencing belongs in the playbook, not improvised mid-incident.
The layer nobody covers
Secure design. Every source here is either a runtime control or an organisational process. Nothing in the corpus addresses threat-modelling a RAG pipeline, deciding what a tool should be permitted to express, or structuring a multi-agent trust boundary before code exists. The ASI reference architecture and its six-step decision path are the closest thing the vault holds, and they are a scoping aid rather than a design method.
CSA’s MAESTRO is the named candidate — a layered threat-modelling methodology for agentic systems — but no primary source for it is held here.
Related pages
- control-catalogue — the controls themselves, by cross-framework agreement
- mitigation-frameworks — how the four control sets compare
- where-defenses-fail — the structural limits
- telemetry-and-logging — the control every layer depends on