Choosing an attacker model

The attacker role is the model that writes the adversarial prompts. PAIR, TAP, PAP, AdvPrefix, AutoDAN-Turbo, Crescendo, FC-Attack, tFC-Attack and RAG poisoning cannot run without one, and Tool-output IPI and ToolHijacker can use one (see Shared settings). Which model fills it is the biggest lever on how many jailbreaks you find, how long each goal takes and how much GPU you need.

A good attacker follows the attack's output format, rarely refuses to write adversarial text, and is strong enough to learn from the target's replies. This page summarizes a measured head-to-head of ten attacker configurations run by HackAgent against one target, so you can pick with evidence instead of guesswork. To skip running your own, use the gateway's hackagent/attacker-v1, described in Shared settings.

Every figure below is interactive: hover any point for the full per-attacker card (ASR, goal counts, thinking mode, params, serving, memory).

Comparison limitation

These numbers come from a single benchmark campaign (OmniSafeBench goal set, 10 goals per subcategory from categories A-F) against target qwen3.8:27b, judged by qwen3.5:9b. The target, goals and judge were kept constant, so use the plots to compare attackers with each other, not as absolute scores for other targets. The percentages will not carry over to another target. The rankings and the thinking and serving effects may, but one target cannot show that.

How the numbers were produced

• Attacks: TAP and Crescendo, pooled
• Denominator: every attacker ran 340 goals per attack, 680 total per attacker
• Scope: 10 attacker variants = 3 model families (think ON/OFF) + 4 Ollama GGUF builds, on three serving setups (vLLM BF16 TP1, vLLM FP8/NVFP4 TP4, Ollama Q4_K).

Figures

Speed vs ASR

The main trade-off view: attack success against mean minutes per goal (log time axis). Top-left is better; the green quadrant is above-median ASR and below-median time.

Thinking OFF → ON

Same models with reasoning on vs off, the campaign's cleanest controlled contrast (identical weights, judge and goals). Arrow labels show the ASR lift in percentage points and the change in minutes per goal when thinking was enabled.

Size (total params)

ASR against total parameters. To show how parameter count affects ASR, every model in this view has thinking enabled.

Memory footprint

ASR against peak VRAM (log axis): vLLM points are weights (cluster total) plus logged KV cache; Ollama GGUF points are weights only, so they might understate real usage.

Headline numbers (TAP+Crescendo, n=680 each)

Attacker	ASR	95% CI	Avg min/goal	Serving
Qwen-3.8 BF16 thinkON	54.1%	50.4–57.8	21.0	vLLM BF16 TP1
GLM-5.3 FP8 thinkON	48.8%	45.1–52.6	25.3	vLLM FP8 TP4
Qwen-3.8 GGUF	48.2%	44.5–52.0	41.9	Ollama Q4_K
GLM-5.3 FP8 thinkOFF	44.7%	41.0–48.5	24.8	vLLM FP8 TP4
Deepseek V4 Flash thinkON	32.9%	29.5–36.5	22.6	vLLM NVFP4 TP4
Gemma-31B GGUF	28.7%	25.4–32.2	46.1	Ollama Q4_K
Qwen-3.8 BF16 thinkOFF	27.7%	24.4–31.1	48.9	vLLM BF16 TP1
Deepseek V4 Flash thinkOFF	19.0%	16.2–22.1	21.9	vLLM NVFP4 TP4
GLM-4.7 GGUF	18.1%	15.4–21.1	47.6	Ollama Q4_K
Qwen-3.5-4B GGUF	6.8%	5.1–8.9	42.5	Ollama Q4_K

Wilson 95% intervals; times are campaign wall-clock per goal including retries.

What to take from it

• Turn thinking on (generation.thinking: true on the role). All three ON/OFF pairs improved with reasoning enabled: Qwen-3.8 BF16 +26.4 pp (and halved time per goal, the attacker stops earlier due to success), Deepseek +13.9 pp, GLM-5.3 +4.1 pp. Cost is essentially zero on the two that were already fast.
• ASR scales with size, but not always. The 4B floor is weak (6.8%), but in the 27–31B band ASR spans 18–54% at identical size, and the 300B+ MoE pair splits 48.8% vs 32.9%. Serving and quantization matter as much as parameter count.

Caveats

1. The highest-scoring attacker, Qwen-3.8, is an abliterated version of the target model, which flatters its score.
2. Single judge, no panel comparison.
3. Ollama points omit KV cache in the memory figure.