Skip to main content

hackagent.orchestrator.campaign.results

What a campaign run produces, and how it is written out.

TRACE_SCHEMA_VERSION​

Version of the trace payload written to the store and the trace file.

Attempt Objects​

@dataclass(frozen=True)
class Attempt()

One request sent for one goal, the target's reply, and its verdict.

An iterative attack reports one of these per exchange it kept, with request_index counting them in the order they were tried, and metadata holding what the attack reported about it (step, technique…). trace covers the whole search and is carried by the first attempt of a goal.

error is set when the attempt produced nothing to judge: request generation or the search failed, or the target returned a provider error.

labels​

The goal's taxonomy labels (category, subcategory), if any.

prompt​

@property
def prompt() -> str

Text of the last user message, the part the judges see.

errors​

@property
def errors() -> tuple[str, ...]

Every problem the attempt hit: its own, the panel's, each judge's.

user_text​

def user_text(messages: Sequence[Mapping[str, Any]]) -> str

Text parts of the last user message.

AttackOutcome Objects​

@dataclass(frozen=True)
class AttackOutcome()

Every attempt of one attack, or the error that stopped it.

run_trace​

Trace of run-scoped setup (an attack's prepare()), belonging to no single goal. Empty unless the attack did such work.

CampaignResult Objects​

@dataclass(frozen=True)
class CampaignResult()

campaign_run_id​

Primary key of the normalized campaign run in storage. None for a dry run, which deliberately creates no storage records.

audit​

What the configured judge audit found, when one ran.

outputs​

Files written by :func:write_outputs.

succeeded​

@property
def succeeded() -> bool

Every attack ran to completion (attempt-level errors aside).

attempt_errors​

@property
def attempt_errors() -> int

Attempts that failed to generate, reach the target, or be judged.

attempt_row​

def attempt_row(attempt: Attempt, output: OutputSpec) -> dict[str, Any]

Flat, JSON-safe view of an attempt, honouring prompt/response retention.

trace_payload​

def trace_payload(node: TraceNode, output: OutputSpec) -> dict[str, Any]

JSON-safe view of a trace node, honouring prompt/response retention.

path is the node's position in the search, so a reader can rebuild the tree from a flat list of these.

trace_step_type​

def trace_step_type(node: TraceNode) -> str

The store's StepKind for a node; the real kind is in the payload.

StepKind is fixed by the tracking API, and the remote backend maps anything it does not know onto OTHER. Model calls are the one kind that has a faithful member.

write_outputs​

def write_outputs(
        result: CampaignResult,
        output: OutputSpec,
        campaign: Optional[Mapping[str, Any]] = None) -> tuple[Path, ...]

Write a run's files into <directory>/<run_id>/.

campaign.yaml is the campaign that ran, so the run can be reproduced even when it was never saved as a file. results.json and/or results.jsonl hold one row per attempt. Search traces of iterative attacks go to traces.jsonl, one line per attempt, so the result rows stay readable, and a judge audit goes to audit.json.

summary​

def summary(result: CampaignResult) -> dict[str, Any]

Short machine-readable report of a run.