Summary: Stealing a model’s functionality, parameters, or training data — usually through the API rather than the filesystem. Named in the OWASP Top 10 for LLMs and in AI Act Article 15(5) as “confidentiality attacks.”
Sources: raw/Model-theft/ (5 files); see individual citations
Last updated: 2026-07-29
What makes it different from ordinary IP theft
Snyk’s framing is the clearest in the corpus:
| Dimension | Model theft |
|---|---|
| Target | ”learned parameters and architectural knowledge rather than raw data” |
| Method | ”API queries instead of direct system infiltration” |
| Goal | ”functionally equivalent models, not exact copies” |
| Detection | ”Appears as legitimate API usage, making it harder to identify” |
(source: Understanding AI Model Theft...md, Snyk)
The third row is the crux. There is no exfiltrated file to catch at the egress point — the attacker rebuilds an equivalent asset from permitted interactions.
The attack families
Consolidating across sources:
Model extraction / query-based — systematically querying to reverse-engineer decision boundaries. The practical, well-documented case: model-extraction.
Model inversion — “reconstruct sensitive training data by exploiting model responses, particularly dangerous for models trained on personal or proprietary datasets” (Snyk). The target is the training data, not the model.
Membership inference — “determining whether a specific data point was used to train the model” (source: Model Theft_ The Essential Guide _ Nightfall...md). A privacy attack more than an IP one, and the one with the most direct GDPR exposure.
Supply chain infiltration — “compromise AI dependencies, injecting backdoors through poisoned packages or compromised model repositories” (Snyk). Overlaps data poisoning — same vector, different objective.
Side-channel and reverse engineering — inferring architecture from timing, power, or other physical leakage (source: Model Theft in AI_ How IP and Models Get Stolen.md, LayerX).
Insider threats and direct breaches — the conventional path to the weights themselves (LayerX).
Alignment-aware extraction — “recent research demonstrates targeted attacks on large language models that exploit alignment mechanisms to extract more detailed model information” (Snyk).
Sophistication vs detectability
Snyk’s comparison, which is useful for prioritisation:
| Attack type | Technical skill | Resources | Detection difficulty |
|---|---|---|---|
| API exploitation | Low-medium | Low | Low |
| Model extraction | Medium | Low-Medium | Medium |
| Model inversion | High | Medium | High |
| Supply chain | High | Low | Very high |
| Alignment-aware | Very high | High | Very high |
Note the inversion between skill and detectability: supply chain attacks need low resources and are very hard to detect. That is the worst quadrant. See Supply Chain Attacks.
Where you are exposed
Snyk names: “insecure API endpoints,” “insufficient query monitoring — lack of real-time tracking for malicious prompts.”
The recurring theme across all five sources is that the API is the exposure. LayerX’s core techniques list leads with “API scraping and query-based attacks.”
Business impact
Beyond the direct IP loss, the sources converge on a second-order harm that matters more for security teams: a stolen model is a white-box laboratory for attacking the original. See model-extraction and automated attack generation.
Where it sits in the frameworks
| Framework | Placement |
|---|---|
| AI Act Art 15(5) | “confidentiality attacks or model flaws” |
| NIST AI RMF | Secure and Resilient — “exfiltration of models, training data, or other intellectual property through AI system endpoints” |
| MITRE ATLAS | AI Model Access tactic — access to the model as an objective distinct from system access |
| OWASP Top 10 for LLMs | LLM10 Unbounded Consumption — model theft merged with DoS in the 2025 list, both defeated by query limits |
| CoSAI | ”Model extraction attempts that steal proprietary AI capabilities” |
The Code of Practice sets a 5-day reporting deadline for a “serious cybersecurity breach (e.g. exfiltration, cyberattack)” — with no Article 73 equivalent. For a GPAI systemic-risk provider, weight exfiltration is a 5-day reportable event. See serious incident reporting.
The two exfiltration paths
This collection and weight-level attacks cover complementary halves:
- The parameter file — insider theft, breach, or open release. Controls: Code of Practice Appendix 4 (encryption, attested TEEs, copy registries, vetting).
- The behaviour — extraction through the API. Controls: rate limiting, output perturbation, watermarking. See model-theft-defenses.
Most organisations invest heavily in the first and not at all in the second, which is exactly the false-security assumption Praetorian identifies.