Summary: Wiki pages for raw/Model-theft/ — stealing a model’s functionality, parameters or training data, usually through the API rather than the filesystem.
Sources: raw/Model-theft/ (5 files, ~7,300 words)
Last updated: 2026-07-29
Pages
| Page | Covers |
|---|---|
| model-theft | Attack families, sophistication vs detectability, framework placement |
| model-extraction | The worked attack — 1,000 queries, 80.8% agreement |
| model-theft-defenses | Rate limiting, output minimisation, watermarking, pattern analysis |
The one sentence
“Behavior is the model. Every query-response pair is a training example for a replica.”
(source:
raw/Model-theft/Stealing AI Models Through the API...md, Praetorian)
Keeping weights private is not sufficient protection. The API hands out the model’s behaviour one query at a time, and that is enough to rebuild a functional equivalent.
Four things to carry
- Architecture knowledge is unnecessary. Praetorian’s replica used a deliberately different, simpler architecture and still reached 80.8% agreement with the victim.
- Replicas inherit failure modes. “Even when both models are wrong, they’re often wrong in the same way” — which makes a stolen model a white-box oracle for attacking the original.
- Soft probabilities are the leak. Confidence scores reveal inter-class relationships. Truncating to top-k or hard labels is the highest-leverage control, and it degrades your product.
- Supply chain is the worst quadrant — low resources, very high detection difficulty (Snyk). See Supply Chain Attacks.
Rate limiting is the convergent control
Four independent sources in the vault land on it: Praetorian (extraction), PITAX T09 and T41 (gradient attacks and best-of-N), and Code of Practice Appendix 4.3 (“output rate limiting on interfaces”). If you implement one thing from this collection, implement this.
Cross-collection links
- Weight-level attacks — the other exfiltration path, plus T58 oracle extraction
- ATLAS AI Model Access tactic — where model access becomes a first-class objective
- AI Act Art 15(5) — “confidentiality attacks”
- Serious incident reporting — exfiltration is a 5-day reportable event under the Code of Practice
- Root index
Gaps
- All five sources are vendor or consultancy content. Only Praetorian’s contains original technical work; the rest are overviews, and Nightfall’s duplicates its own sections verbatim.
- No source covers extraction against LLMs specifically — the worked example is a CNN classifier. LLM extraction economics differ substantially.
- Model theft is folded into OWASP LLM10 Unbounded Consumption; exposed inference endpoints are also treated as a supply-chain exit in supply chain attacks.
Log
See log.