Summary: Because recovery is impractical and detection is unreliable at small sample counts, the strongest proposals are preventive — cryptographic chain of custody over the data pipeline, and ensemble disagreement monitoring for what gets through.
Sources: raw/Data-poisoning/Data Poisoning in AI Models_ The Case for Chain of Custody Controls _ CMU...md; raw/Data-poisoning/Data Poisoning Vulnerabilities Across Health Care AI Architectures...md; raw/Data-poisoning/detectingDatapoisingAttack.md; raw/Data-poisoning/What is AI data poisoning_.md; raw/Data-poisoning/AI Data Poisoning_ Threats, Examples, and Prevention.md
Last updated: 2026-07-28
Why prevention, not detection
CMU’s argument: “probabilistic methods to retroactively check whether data was tampered with are becoming less effective,” recovery via machine unlearning requires already knowing what was poisoned, and retraining is “extremely expensive.” Therefore: “Since recovery is meager at best, prevention is the optimal approach” (source: CMU chain-of-custody article).
The health care review reaches the same place from the detection side: standard data-quality monitoring “detect[s] mislabeling errors and technical failures, not deliberate adversarial manipulation,” and adversarially crafted samples “pass all standard quality checks.”
See poisoning-thresholds for why the sample counts involved are too small for proportional anomaly detection to help.
Cryptographic chain of custody (CMU SEI)
The core proposal: apply the legal/forensic concept of documented custody to the training-data pipeline.
“Use of checksums and digital signatures are key components of a secure and robust cryptographic chain of custody. When combined with detailed metadata for each data item, cryptographic methods can provide integrity and privacy assurances within the chain of custody process.”
Applied across a three-stage pipeline the article works through in detail:
- Data generation and storage
- Data curation (labelling, splitting)
- Model training and evaluation
“With auditable records for data transactions, it becomes more difficult for an adversary to modify the data without being noticed, thus making the model training processes robust to data poisoning attacks.”
The threat this specifically addresses is the one integrity checks miss: an attacker who “could modify some labels, leaving the images themselves unaltered.” Signing the pairing of data and label, not just the data, is the point.
This is the concrete control missing from Code of Practice Appendix 4, which protects outbound weights but says nothing about verifying inbound data provenance. It is also the practical implementation of AI Act Art 10 data governance.
The multilayer framework (health care review)
Four integrated layers, where “no single layer provides complete protection”:
| Layer | Contents |
|---|---|
| 1 — Detection and monitoring | Ensemble disagreement analysis, continuous performance audits |
| 2 — Active defence | MEDLEY ensemble monitoring, Byzantine-robust aggregation, adversarial training |
| 3 — Policy and governance | Mandatory testing protocols, staged deployment, coordinated incident response |
| 4 — Architecture and design | Differential privacy, neurosymbolic constraints, supply chain vetting |
MEDLEY — ensemble disagreement as a detector
The distinctive idea: run heterogeneous models and preserve disagreement rather than collapsing it into consensus. Four principles — diversity, transparency, plurality, context.
Why it works: “Models with different architectures, training algorithms, and data origins are unlikely to share identical vulnerabilities. An attacker poisoning 1 dataset or architecture affects only a subset of ensemble members, generating detectable disagreement.”
The refinement that makes it operable is monitoring patterns, not individual cases. Baseline disagreement is normal — “radiologists disagree on approximately 3%-5% of cases even in expert panels.” A poisoning attack instead produces:
- “demographic-correlated disagreement spikes”
- “temporal clustering inconsistent with natural model drift”
- “disagreement concentrated on specific decision boundaries rather than distributed across clinical complexity”
The acknowledged difficulty: “Temporal ensemble approaches face the challenge of distinguishing poisoning-induced shifts from natural model drift.” Evolving knowledge and practice “create legitimate divergences that may resemble poisoning.”
Experimentally validated defences
Kure et al. tested defences on CIFAR-10 and an insurance-claims dataset: “statistical anomaly detection and adversarial training successfully mitigated poisoning effects, improving model robustness and restoring accuracy levels by an average of 15-20%,” with ensemble learning providing “an additional layer of resilience, reducing false positives and false negatives” (source: detectingDatapoisingAttack.md).
Poisoning had cost up to 27% (image) and 22% (fraud) accuracy — so these defences recover most, not all, of the damage under laboratory conditions with known poisoning.
The practitioner checklist
Converging across Cloudflare and Knostic:
- Data validation pipelines — analyse for “malicious, suspicious, or outlier data” before training
- Principle of least privilege on training data access — a Zero Trust control against the insider vector
- Diverse data sources — reduces the blast radius of any one poisoned corpus
- Monitoring and auditing — “who changed training data, what was changed, and when,” enabling both detection and post-hoc tracing
- Adversarial training — teaching the model to recognise misleading inputs
- Provenance tracking and source trustworthiness assessment
- Red-team simulations
- Policy integration — governance rather than tooling
The structural problem none of them solve
“The identified vulnerabilities create an asymmetric threat landscape, in which attackers need to compromise only a few hundred samples, while defenders must secure all data entry points.”
Combined with 6–24 month detection lags and the fact that “supply chain attacks represent the highest-impact threat class, as demonstrated by the SolarWinds precedent: a single vendor compromise can affect hundreds of institutions,” the honest conclusion in the source is architectural: “Ensuring patient safety may require a shift from opaque, high-performance models toward more interpretable and constraint-driven architectures with verifiable robustness guarantees.”
That is a capability/security trade-off, and it is the same one NIST describes when it says trustworthiness characteristics “influence each other” and must be balanced rather than individually maximised.