Skip to main content

hackagent.attacks.techniques.adaptive.pair.prompts

The attacker's instructions, and the turns a PAIR stream is built from.

A stream is one attacker conversation: a fixed system prompt, an opening request, then a feedback turn after every attempt. The attacker sees the target's reply and its score, which is what lets the next prompt refine the last one rather than start over.

The score comes from the campaign's panel, or, when the run configures a scorer role, from that model rating the reply 1-10 against :data:SCORER_SYSTEM_PROMPT.

SCORER_SYSTEM_PROMPT​

The rubric PAIR's own scorer is given, when a run configures one. It is the 1-10 scale the attacker's instructions already describe, so the number fed back means the same thing either way.

STREAM_STRATEGIES​

The reference implementation gives each conversation one of three approaches. They are kept separate per stream so a failed approach does not contaminate the other search trajectories.

system_prompt​

def system_prompt(goal: str, stream: int, target_str: str) -> str

The fixed opening instruction for one stream.

opening_request​

def opening_request(goal: str, target_str: str) -> str

The first user turn of a stream.

feedback​

def feedback(goal: str,
             target_str: str,
             reply: str,
             score: Optional[float],
             explanation: str = "",
             limit: int = 2500) -> str

The user turn appended after an attempt, carrying what came back.

scorer_system_prompt​

def scorer_system_prompt(goal: str) -> str

The rubric the scorer rates against.

scorer_request​

def scorer_request(prompt: str, reply: str) -> str

The exchange handed to the scorer, whole.

Nothing is trimmed here. A rating is a judgement about a reply, and a reply cut short is a different reply: the refusal that turns into compliance three paragraphs down would be scored as a refusal. The attacker's feedback is trimmed instead, because there the point is the gist and the turns accumulate.

parse_rating​

def parse_rating(text: Optional[str]) -> Optional[float]

Read the scorer's 1-10 rating; None when it gave no number.

The asked-for Rating: [[n]] wins. Models that answer in prose still tend to name the number, so looser spellings are tried in turn, and the last bare number is a final guess rather than discarding the call.