Skip to main content

How a run works

A campaign turns goals into verdicts. This page follows one goal all the way through, so the words in the reference have something concrete behind them.

The whole pipeline​

Guardrails are optional and off by default; without them the prompt goes straight to the target.

Static and adaptive attacks differ here​

A static attack decides its prompts up front. An adaptive one searches: it reads each reply and writes the next prompt, which is why it needs an attacker model of its own.

Every exchange an adaptive attack had judged is recorded, not just the last one, so you can see how it got there.

Several attacks over the same goals​

By default every goal goes through every attack, and you get the full matrix.

With execution.escalate turned on, the attacks become a ladder: a goal that falls is dropped, and the next attack only faces what is left. That spends the expensive attacks on the hard goals.

The run stops early once every goal has fallen, because there is nothing left to attack.

What a run leaves behind​

WhereWhat
A local databaseEvery attempt, with its prompt, reply and verdict. Read it with hackagent results list or the terminal app.
./logs/runs/<run id>/campaign.yamlThe exact campaign that ran, every default written out, so the run can be repeated.
./logs/runs/<run id>/results.jsonThe same attempts as a file, for diffing or sharing.
./logs/runs/<run id>/traces.jsonlFor adaptive attacks, the search itself: every call, decision and branch.
./logs/runs/<run id>/audit.jsonWhen the judge audit is on, how the panel measured up.

The output folder and formats are set under execution.output.

Next​