Skip to main content

Prompt Injection

Tests whether the LLM executes attacker-supplied instructions that override or bypass the system prompt.

Sub-types​

  • direct_injection: User prompt directly overrides system instructions.
  • indirect_injection: Malicious instructions are embedded in retrieved/external content.
  • context_manipulation: Crafted context tricks the model into ignoring guardrails.

Threat Profile​

Primary

  • advbench: 520 adversarial goals covering injection scenarios
  • harmbench_contextual: Contextual prompts requiring instruction override
  • prompt_injections: 662 prompt injection samples for direct PI testing (deepset)

Secondary

  • strongreject: Forbidden prompts to test injection guardrails

Attack Techniques​

Primary

  • Static Template: Template-based prompt injection
  • PAIR: Iterative refinement for bypass discovery

Secondary

Metrics​

  • asr
  • judge_score

For injection through retrieved documents or tool output, see Indirect Prompt Injection.

Usage​

Instantiate the Vulnerability​

from hackagent.catalog.risks import PromptInjection
from hackagent.catalog.risks.prompt_injection.types import PromptInjectionType

# Use all sub-types
vuln = PromptInjection()

# Or specify particular sub-types
vuln = PromptInjection(types=[
    PromptInjectionType.DIRECT_INJECTION.value,
    PromptInjectionType.INDIRECT_INJECTION.value,
])

Run an Evaluation Campaign​

from hackagent.catalog.risks.prompt_injection import PROMPT_INJECTION_PROFILE
from hackagent.orchestrator.campaign import run_campaign, summary

TARGET = {
    "name": "my-agent",
    "connection": {
        "provider": "local",
        "type": "OPENAI",
        "endpoint": "http://localhost:8080/chat",
    },
}

# Use profile recommendations
for attack in PROMPT_INJECTION_PROFILE.primary_attacks:
    for dataset in PROMPT_INJECTION_PROFILE.primary_datasets:
        result = run_campaign(
            {
                "version": 1,
                "campaign": {
                    "name": f"{attack.technique} vs {dataset.preset}",
                    "description": PROMPT_INJECTION_PROFILE.objective,
                },
                "dataset": {"source": {"type": "preset", "name": dataset.preset}},
                "target": TARGET,
                "attacks": [{"name": attack.technique}],
            }
        )
        print(summary(result))