Summary: Rate limiting, output minimisation, watermarking and query-pattern analysis. Every one trades product quality or cost against security, and the sources are unusually honest about it.
Sources: raw/Model-theft/ — Praetorian, Snyk, LayerX, F5, Nightfall
Last updated: 2026-07-28
The two defensive postures
Nightfall’s split is the useful frame:
- Reactive — “detecting and mitigating model theft attacks after they have occurred,” e.g. input sanitisation.
- Proactive — “designing machine learning models that are robust to model theft attacks,” e.g. adversarial training.
(source: Model Theft_ The Essential Guide _ Nightfall...md)
Controls that reduce information per query
These attack the extraction economics directly by making each query worth less:
Prediction truncation — “Return only top-k classes or hard labels instead of full probability distributions. Less information per query means more queries required for extraction” (Praetorian).
Output perturbation — “Add calibrated noise to confidence scores. This degrades the information available for distillation while maintaining utility for legitimate users” (Praetorian).
Both follow from the finding that “soft probability outputs contain far more information than hard labels.” They are the highest-leverage controls available, and both degrade the product. Confidence scores exist because customers want them. This is a genuine trade, not a free win. See model-extraction.
Controls that limit query volume
Rate limiting — “Restrict query volume per user to prevent the bulk data collection needed for training replica models” (Praetorian). Their attack used 1,000 queries, “a number that should trigger anomaly detection.”
Rate limiting recurs across the vault as the control that bites hardest on query-hungry attacks:
- PITAX T09 gradient-based attacks need “the ability to send thousands of queries,” making them “most applicable to self-hosted models, controlled pentesting environments… rather than rate-limited SaaS APIs.”
- Best-of-N (T41) scales “as a power law with N” — the attack has a price, and rate limits set it.
- Code of Practice Appendix 4.3 requires “output rate limiting on interfaces” specifically to prevent exfiltration.
Four independent sources land on the same control. If you do one thing, do this.
Controls that detect the pattern
Behavioural monitoring — flag “rapid uploads of diverse images, sequential queries with slight variations, or systematic coverage of the input space” (Praetorian).
Query pattern analysis — “Legitimate users have characteristic query patterns. Extraction attacks look different, more systematic, more diverse, more evenly distributed across the input space” (Praetorian).
The signal is coverage, not volume. An extraction campaign systematically samples the input space in a way real usage never does. This is the same shift in unit-of-analysis that multi-turn attacks force — the individual request is innocuous; the campaign is not.
Controls that establish provenance after the fact
Watermarking — “Embed detectable patterns in model behavior that survive extraction. If a stolen model appears in the wild, watermarks provide evidence of theft” (Praetorian).
Watermarking does not prevent anything. It converts an undetectable theft into a provable one, which shifts the response from security to legal. Worth noting it is the same technical family as the machine-readable marking AI Act Art 50(2) mandates for synthetic content — and it inherits the same robustness problem, since removal is an adversarial goal.
Access-layer controls
From LayerX and F5, both vendor-framed but structurally sound:
- Secure API and access controls — authentication, authorisation, least privilege on endpoints
- Bot defence — distinguishing automated from human query traffic
- API security gateways and WAF-style enforcement in front of inference endpoints
- Preventing malicious data exfiltration at the egress point
F5’s framing is “defence in depth: a combined approach.” No single control is claimed to be sufficient, which is the correct posture given extraction needs only legitimate access.
Snyk’s technical layers
“Access control and rate management” plus “advanced protection techniques,” with a comparison of defence mechanisms. Snyk also stresses detection and response — “knowing when you’re under attack” — reinforcing that extraction is not fully preventable and must be monitored for.
What none of these do
They protect the deployed API. They do nothing once weights are released or stolen outright — at that point abliteration strips safety behaviour and no model-level control survives.
That is why the corpus treats weight confidentiality and behavioural confidentiality as separate problems with separate control sets:
| Path | Controls | Source |
|---|---|---|
| Parameter file | Encryption, attested TEEs, copy registries, vetting, physical access | CoP Appendix 4 |
| Behaviour via API | Rate limits, truncation, perturbation, watermarking, pattern analysis | this page |
Neither substitutes for the other, and Appendix 4 — the most prescriptive control set in the vault — covers only the first.