Skip to main content

AI Risks

Start here when you know what you are worried about rather than which attack to run. A names one way an AI system can fail. Its threat profile picks the datasets, attacks and metrics for testing it, and a runs them.

Built-in risks​

Jailbreak · Prompt Injection · System Prompt Leakage · Input Manipulation Attack · Model Evasion · Craft Adversarial Data · Sensitive Information Disclosure · Misinformation · Excessive Agency · Malicious Tool Invocation · Credential Exposure · Public-Facing Application Exploitation · Vector and Embedding Weaknesses Exploit

Vulnerabilities describes each in one line. Retrieval-based systems also have a dedicated indirect prompt injection scenario, and you can define your own risks.

Test a risk​

A campaign runs the Jailbreak profile's primary attacks (h4rm3l, TAP and PAIR) on HarmBench, one of its recommended datasets. With escalate, TAP and PAIR only face the goals h4rm3l did not break:

jailbreak.yaml
version: 1
campaign:
name: Jailbreak profile
dataset:
preset: harmbench
selection:
limit: 20
target:
name: my-agent
connection: {provider: vllm, type: OPENAI, endpoint: http://localhost:8000/v1}
attacks:
- name: h4rm3l
- name: tap
roles:
attacker: {name: llama3.2, connection: {provider: ollama, type: OLLAMA}}
- name: pair
roles:
attacker: {name: llama3.2, connection: {provider: ollama, type: OLLAMA}}
evaluation:
judges:
- name: llama3.2:latest
connection: {provider: ollama, type: OLLAMA}
scoring: {type: harmbench}
execution:
escalate: true
hackagent campaign run jailbreak.yaml

Each risk's page lists its datasets, attacks and metrics. Evaluation campaigns covers larger runs: quick scans, full audits and targeted assessments. To run one, start with Your first campaign; the attack catalog lists every technique.