Summary: Four PITAX techniques that replace the human attacker with a search process — gradient descent, mutation fuzzing, best-of-N sampling, or a self-improving agent with a strategy library.
Sources: raw/AI-attack-techniques/PIT-T-09,39,40,41-*.md
Last updated: 2026-07-28
These techniques do not introduce new evasions. They search the space of existing ones, which changes the economics: attack cost drops from human creativity to compute.
The four search strategies
T09 Gradient-Based Attacks — “Automated adversarial token discovery using gradient descent or iterative optimization to find inputs that flip model responses” (source: PIT-T-09-gradient-based-attacks.md; Universal Adversarial Triggers, Wallace et al. arXiv 1908.07125; Adversarial Suffix, OWASP LLM01:2025; “Optimization-based attacks” per NIST).
Critically scoped: these “require either white-box model access (for gradients) or the ability to send thousands of queries, making them most applicable to self-hosted models, controlled pentesting environments, or on-premise deployments rather than rate-limited SaaS APIs” (source: PIT-T-09). Rate limiting is a real control here, not theatre.
T39 Fuzzing-Based Jailbreak — “mutation-based search that starts from seed jailbreak templates and applies operators (generate, crossover, expand, shorten, rephrase) guided by a judge model to evolve high-success templates. The black-box analog of software fuzzing for prompt attacks” (source: PIT-T-39-fuzzing-based-jailbreak.md; GPTFUZZER arXiv 2309.10253; FuzzLLM arXiv 2309.05274; TurboFuzzLLM arXiv 2502.18504).
T41 Best-of-N — “repeatedly samples random augmentations of a prompt, character scrambling, random capitalization, character noising (and image/audio analogues), until one variant slips past safety. Attack success scales as a power law with N. Composes existing evasions rather than introducing a new encoding” (source: PIT-T-41-best-of-n-augmentation-sampling.md; Anthropic arXiv 2412.03556).
T40 Autonomous Strategy Discovery — “a black-box, lifelong-learning agent that discovers jailbreak strategies from scratch, stores them in a growing strategy library, and recombines/evolves them with no human-authored seeds, then applies test-time scaling (best-of-N, beam search) over the library” (source: PIT-T-40-autonomous-strategy-discovery-autodan-turbo.md; AutoDAN-Turbo arXiv 2410.05295; AutoDAN-Reasoning arXiv 2510.05379).
The power law is the finding
T41’s stated result — success scales as a power law with N — means there is no N at which the attack stops working, only an N you have not paid for. A guardrail with a nonzero false-negative rate under random augmentation is a guardrail with a known price.
This reframes evaluation: “our filter blocks this jailbreak” is not a claim about the filter unless it comes with the sampling budget it survived.
Cost, not capability, is the defensive variable
Three of these four are black-box (T39, T40, T41). They need only query access. The defensive levers that actually bite are the ones that raise per-query cost or cap query volume:
- Rate limiting and per-account query budgets (the explicitly stated limiter for T09).
- Detecting the search pattern — high-volume near-duplicate prompts, systematic augmentation — rather than any individual prompt. Every individual query in a BoN run is, by construction, one that the filter’s decision boundary is being probed with.
Why T40 is the different one
T39 and T41 search a fixed space. T40 accumulates a strategy library across runs and recombines it “with no human-authored seeds.” It is the only technique here whose capability grows between engagements. A defence measured against it decays even if nothing about the model changes.
Governance hook
The GPAI Code of Practice requires signatories to conduct and document “adversarial testing” and “rigorous model evaluations” including “model elicitation” against systemic risk, using “Qualified Evaluation Teams” and independent external evaluations (source: raw/Laws/Overview of the Code of Practice...md, Appendix 3). AI Act Article 55(1)(a) makes documented adversarial testing a legal obligation for GPAI models with systemic risk (source: raw/AI-regulations/Regulation - EU - 2024_1689.md).
Automated harnesses of this kind are how that obligation gets discharged at scale.