Summary: The AI Act’s security clause for high-risk systems. Notable because it names specific AI attack classes — data poisoning, model poisoning, adversarial examples, confidentiality attacks — as things providers must engineer against.
Sources: raw/AI-regulations/Regulation - EU - 2024_1689.md (Article 15)
Last updated: 2026-07-28
What it requires
Art 15(1) — high-risk AI systems “shall be designed and developed in such a way that they achieve an appropriate level of accuracy, robustness, and cybersecurity, and that they perform consistently in those respects throughout their lifecycle” (source: Regulation - EU - 2024_1689.md, emphasis added).
Lifecycle, not launch. This is a continuing obligation, which links it to post-market monitoring (Art 72) and serious incident reporting (Art 73).
Art 15(3) — accuracy levels and metrics must be declared in the instructions for use.
Art 15(4) — systems “shall be as resilient as possible regarding errors, faults or inconsistencies,” achievable “through technical redundancy solutions, which may include backup or fail-safe plans.”
Also in 15(4), a specific requirement for continuously-learning systems: they must be developed “as to eliminate or reduce as far as possible the risk of possibly biased outputs influencing input for future operations (feedback loops), and as to ensure that any such feedback loops are duly addressed with appropriate mitigation measures.”
The security paragraph
Art 15(5) is the operative security text:
High-risk AI systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities.
The technical solutions aiming to ensure the cybersecurity of high-risk AI systems shall be appropriate to the relevant circumstances and the risks.
The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training data set (data poisoning), or pre-trained components used in training (model poisoning), inputs designed to cause the AI model to make a mistake (adversarial examples or model evasion), confidentiality attacks or model flaws.
(source: Regulation - EU - 2024_1689.md, Article 15(5))
Why this article matters more than its length suggests
Four things are unusual about it:
- It names attack classes in binding law. Most cybersecurity regulation stops at “appropriate technical and organisational measures.” Article 15(5) enumerates five specific AI attack categories.
- It requires the full response lifecycle — “prevent, detect, respond to, resolve and control for” — not just prevention. That is an incident-response obligation embedded in a design requirement. See AI incident response.
- “Where appropriate” is the escape valve. The enumerated measures are qualified. Expect the substance to be set by harmonised standards and by what counts as “generally acknowledged state of the art” under Art 8(1).
- Art 15(2) hands measurement to standards bodies: the Commission “shall… encourage, as appropriate, the development of benchmarks and measurement methodologies” with metrology and benchmarking authorities. Absent benchmarks, “appropriate level of robustness” is currently unmeasurable in any agreed way.
Mapping to the attack corpus
The Act’s five categories map onto the PITAX taxonomy unevenly:
| Art 15(5) category | Where this vault covers it |
|---|---|
| Data poisoning | [[../data-poisoning-wiki/index |
| Model poisoning | [[../data-poisoning-wiki/data-poisoning |
| Adversarial examples / evasion | [[../ai-attack-techniques-wiki/automated-attack-generation |
| Confidentiality attacks | [[../model-theft-wiki/index |
| Model flaws | [[../ai-attack-techniques-wiki/obfuscation-and-encoding |
A caution on “detect” for data poisoning. The newer evidence is that poisoning success depends on the absolute number of poisoned samples rather than their share of the corpus — 100–500 samples across every architecture studied, with detection lags of 6–24 months. If that holds, proportional anomaly detection cannot satisfy the “detect” limb at any dataset scale, and compliance has to rest on provenance controls and Art 10 data governance instead. The sources in this vault disagree on this point; see poisoning thresholds.
The notable gap: prompt injection is not named. The Art 15(5) list is drawn from the classical adversarial-ML literature and predates the agentic threat model. Nothing in it obviously covers tool-definition injection or indirect injection. Those would have to be read into “attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities” in 15(5) first subparagraph — which is broad enough to carry them, but does so by general language rather than by name.
By contrast, the GPAI Code of Practice does address the agentic threat model directly, listing “cyber offence” as a specified systemic risk and requiring sandboxing against model self-exfiltration (source: raw/Laws/Overview of the Code of Practice...md). See Code of Practice security mitigations.
Frameworks that fill the “how”
Article 15 says what; it does not say how. The corpus supplies three candidates:
- NIST AI RMF — Secure and Resilient is one of seven trustworthiness characteristics; MEASURE and MANAGE supply the process. See NIST AI RMF.
- MITRE ATLAS — threat matrix mapping, recommended for exactly this purpose (source:
raw/Laws/What Is AI Governance_Paloaltonenetworks.md), and now ingested: see MITRE ATLAS and its technique-level mitigations. Note ATLAS tags 114 techniques as Agentic AI — the surface Art 15(5) does not name at all. - Code of Practice Appendix 4 — concrete controls, though scoped to GPAI with systemic risk rather than high-risk systems.
Applicability
Art 15 is in Chapter III Section 2, so it applies from 2 August 2026, except for Art 6(1)-route high-risk systems (product safety components), which apply from 2 August 2027. See ai-act-timeline.
Related pages
- high-risk-requirements — Arts 9–15 in full
- serious-incident-reporting
- AI Attack Techniques
- NIST AI RMF