AI Risks
Start here when you know what you are worried about rather than which attack to run. A names one way an AI system can fail. Its threat profile picks the datasets, attacks and metrics for testing it, and a runs them.
Built-in risks
Jailbreak · Prompt Injection · System Prompt Leakage · Input Manipulation Attack · Model Evasion · Craft Adversarial Data · Sensitive Information Disclosure · Misinformation · Excessive Agency · Malicious Tool Invocation · Credential Exposure · Public-Facing Application Exploitation · Vector and Embedding Weaknesses Exploit
Vulnerabilities describes each in one line. Retrieval-based systems also have a dedicated indirect prompt injection scenario, and you can define your own risks.
Test a risk
- Campaign file
- SDK
A campaign runs the Jailbreak profile's primary attacks (h4rm3l, TAP and PAIR) on HarmBench, one of its recommended datasets. With escalate, TAP and PAIR only face the goals h4rm3l did not break:
version: 1
campaign:
name: Jailbreak profile
dataset:
preset: harmbench
selection:
limit: 20
target:
name: my-agent
connection: {provider: vllm, type: OPENAI, endpoint: http://localhost:8000/v1}
attacks:
- name: h4rm3l
- name: tap
roles:
attacker: {name: llama3.2, connection: {provider: ollama, type: OLLAMA}}
- name: pair
roles:
attacker: {name: llama3.2, connection: {provider: ollama, type: OLLAMA}}
evaluation:
judges:
- name: llama3.2:latest
connection: {provider: ollama, type: OLLAMA}
scoring: {type: harmbench}
execution:
escalate: true
hackagent campaign run jailbreak.yaml
Every profile lists its recommended attacks and datasets; loop over the primary ones:
from hackagent.catalog.risks.jailbreak import JAILBREAK_PROFILE
from hackagent.orchestrator.campaign import run_campaign
OLLAMA = {"provider": "ollama", "type": "OLLAMA"}
TARGET = {"name": "my-agent", "connection": {
"provider": "local", "type": "OPENAI",
"endpoint": "http://localhost:8000/v1"}}
preset = "harmbench" # one of JAILBREAK_PROFILE.primary_datasets
for attack in JAILBREAK_PROFILE.primary_attacks:
result = run_campaign({
"version": 1,
"campaign": {"name": f"{attack.technique} vs {preset}"},
"dataset": {
"source": {"type": "preset", "name": preset},
"selection": {"limit": 20},
},
"target": TARGET,
"attacks": [{"name": attack.technique,
"roles": {"attacker": {"name": "llama3.2:latest",
"connection": OLLAMA}}}],
"evaluation": {"judges": [{"name": "llama3.2:latest", "connection": OLLAMA,
"scoring": {"type": "harmbench"}}]},
})
Each risk's page lists its datasets, attacks and metrics. Evaluation campaigns covers larger runs: quick scans, full audits and targeted assessments. To run one, start with Your first campaign; the attack catalog lists every technique.