Runs and reports
Runs: a campaign run (a parent run) and its attack runs (child runs). Read their status and results, cancel and delete them, read the campaign run reports the dashboard shows (summary, goals, matrix, attempts, costs, comparison) and, for executors, the claim, heartbeat, report and finish protocol.
List runs
Returns the runs of the caller's organization, newest first, with their result counts instead of their results. Campaign runs and their attack runs are listed alike: `parent__isnull=true` keeps campaign runs only, `parent=<id>` lists the attack runs of one. Paginated with `page` and `page_size` (default 10, at most 100). Staff users see every organization's runs. The dashboard's campaign run list is `GET /run/summary`.
Retrieve a run
Returns one run of the caller's organization with its results and their traces, or, with `include_results=false`, with its result counts instead (much lighter for large runs). A cloud run's token may read the runs of its own run tree. For a campaign run's progress and outcome, `GET /run/{id}/summary` and `GET /run/{id}/report` are lighter and computed for you.
Annotate a run
Updates a run's `run_notes`, or merges top-level keys into its `run_config`. Only the run's executor may change `run_config`, and never with a literal credential. Every other field is read-only here: the status changes through `cancel` and `finish`. A cloud run's token may annotate the runs of its own run tree.
Delete a run
Deletes a run with its results and traces. A campaign run is deleted with all its attack runs, and only when it and all of them are `COMPLETED`, `FAILED` or `CANCELLED` (409 otherwise: cancel it first). This cannot be undone. To delete several campaign runs at once, use `POST /run/delete`.
List a goal's attempts
Returns the attempts made on one goal of a campaign run (`goal_index`, required), in order, across its attack runs or one of them (`run_id`), with the page holding the first successful jailbreak. Paginated with `page` and `page_size` (at most 100). Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.
Cancel a campaign run
Cancels a campaign run: a queued one is `CANCELLED` at once; for a running one a cancel is requested (`cancel_requested_at`), and its executor stops and finishes it as `CANCELLED` at its next heartbeat. 400 for an attack run (cancel its campaign run), 409 when the run already finished.
Create an attack run
Executor protocol. Creates a child run (one attack) of a running campaign run, for its executor: `RUNNING`, with the campaign run's executor, organization, owner and campaign, and the given attack configuration and agent (defaults to the campaign run's). The attack and agent must belong to the run's organization. 400 when the run named is itself an attack run, 409 when the campaign run is not running.
Claim a queued run
Executor protocol. Claims a queued campaign run for the caller, which then executes it: a `local` run with an API key or login token of the run's organization (`hackagent run <id>` does this), a `cloud` run with the cloud worker's token only. The run becomes `RUNNING`. A cloud claim also returns the run's token (`run_token`) and the values of the secrets its spec references (`env`); a cloud run that can no longer run (its spec, endpoints or secrets changed) is failed instead. 409 when the run is no longer queued, says who has it, or when its organization has no free cloud slot.
Compare a campaign run with a baseline run
Compares the goal verdicts of this campaign run with those of another visible campaign run (`baseline`, required), matching goals by their text: each goal is `regressed` (now jailbroken), `fixed`, `unchanged`, `new` or `dropped`, with the totals. Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.
Get a campaign run's costs
Returns what each attack run of a campaign run spent: model calls by role (target, attacker, judge), the target's tokens, elapsed time and the target's latency, with per-goal samples (calls and time to the first jailbreak). Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.
Finish a run
Executor protocol. Ends the run as `COMPLETED`, `FAILED` or `CANCELLED`, with `note` as its notes. Its child runs that are still open end the same way, and finishing a campaign run revokes its run token. 409 when the run already finished. Not rate limited.
List a campaign run's goals
Returns the goals of a campaign run with their verdict and each attack run's outcome on them, ordered by goal index. Filter by `goal_index`, `category`, `subcategory`, `verdict` or a `search` in the goal text, and narrow to one attack run with `run_id`. Paginated with `page` and `page_size` (at most 1000). Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.
Send an executor heartbeat
Executor protocol. Reports that the executor of the run's campaign run is alive; any run of the tree may be named. A running campaign run whose executor sends no heartbeat for 5 minutes is failed. The answer says whether a cancel was requested, so the executor can stop and finish the run as `CANCELLED`. Not rate limited.
Get a campaign run's goal matrix
Returns every goal's verdict and attempt count in each attack run, as columnar arrays (goals by attack runs) for heat maps. Never paginated. Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.
Get a campaign run's report
Returns the full report of a campaign run: its summary and spec, each attack run with its own outcomes, the breakdowns by technique and by goal category, the judge audit, and the summary of the latest earlier completed run of the same campaign. With `run_id`, the outcomes cover that attack run only. Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.
Record a result in a run
Records a result in this run, as `POST /result` does, with the run taken from the path (any `run` in the body is replaced). The run must belong to the caller's organization; a cloud run's token may write to its own run tree only.
Get a goal's attack structure
Returns the whole recorded search of one attack run on one goal: its attempts with their metadata, their trace steps and their results, to draw how the attack progressed. Both `goal_index` and `run_id` are required. Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.
Get a campaign run's summary
Returns one campaign run's status and goal verdicts (about 1 KB): what a live view polls while the run executes. Its `revision` moves whenever any report of the run changes, so poll this and refetch the heavier reports only when it moves. Attack runs are 404: name their campaign run. Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.
Claim the next cloud run
Executor protocol, cloud worker token only. Claims the oldest queued cloud campaign run of an organization below its limit of concurrent cloud runs, as `POST /run/{id}/claim` does. Runs the cloud can no longer run are failed and skipped; runs whose endpoints cannot be checked right now stay queued and are skipped. 204 when there is nothing to claim. Not rate limited.
Delete campaign runs
Deletes 1 to 100 campaign runs at once, with their attack runs, results and traces: all of them or none. 404 if any id is not a campaign run visible to the caller, 409 if any of them or of their attack runs is still `PENDING` or `RUNNING` (cancel them first). This cannot be undone. Accepts a login token or an API key.
List campaign runs with their verdicts
Returns the caller's campaign runs (parent runs only), newest first, each with its goal verdicts over all its attack runs: how many goals were jailbroken, mitigated or are unresolved. Filter by `campaign`, `agent` or `status`; a malformed id matches nothing. Paginated with `page` and `page_size` (at most 100). This is the dashboard's run list. Accepts a login token or an API key (not a run token). Every response has a weak `ETag`: send it back as `If-None-Match` to get `304` without a body while nothing changed.