Skip to main content

Vector and Embedding Weaknesses Exploit

Tests for embedding inversion, vector database poisoning, and similarity search manipulation in RAG pipelines.

Sub-types​

  • embedding_inversion: Recovering original text or sensitive data from embedding vectors.
  • vector_db_poisoning: Injecting malicious entries into vector databases used for RAG.
  • similarity_manipulation: Crafting inputs to manipulate similarity search results.

Threat Profile​

Primary

  • rag_security: RAG security benchmarks for vector embedding attacks

Secondary

  • saladbench: SaladBench for broader RAG attack coverage

Attack Techniques​

Primary

Metrics​

  • asr
  • judge_score

Usage​

Instantiate the Vulnerability​

from hackagent.catalog.risks import VectorEmbeddingWeaknessesExploit
from hackagent.catalog.risks.vector_embedding_weaknesses_exploit.types import VectorEmbeddingWeaknessesExploitType

# Use all sub-types
vuln = VectorEmbeddingWeaknessesExploit()

# Or specify particular sub-types
vuln = VectorEmbeddingWeaknessesExploit(types=[
    VectorEmbeddingWeaknessesExploitType.EMBEDDING_INVERSION.value,
    VectorEmbeddingWeaknessesExploitType.VECTOR_DB_POISONING.value,
])

Run an Evaluation Campaign​

from hackagent.catalog.risks.vector_embedding_weaknesses_exploit import VECTOR_EMBEDDING_WEAKNESSES_EXPLOIT_PROFILE
from hackagent.orchestrator.campaign import run_campaign, summary

TARGET = {
    "name": "my-agent",
    "connection": {
        "provider": "local",
        "type": "OPENAI",
        "endpoint": "http://localhost:8080/chat",
    },
}

# Use profile recommendations
for attack in VECTOR_EMBEDDING_WEAKNESSES_EXPLOIT_PROFILE.primary_attacks:
    for dataset in VECTOR_EMBEDDING_WEAKNESSES_EXPLOIT_PROFILE.primary_datasets:
        result = run_campaign(
            {
                "version": 1,
                "campaign": {
                    "name": f"{attack.technique} vs {dataset.preset}",
                    "description": VECTOR_EMBEDDING_WEAKNESSES_EXPLOIT_PROFILE.objective,
                },
                "dataset": {"source": {"type": "preset", "name": dataset.preset}},
                "target": TARGET,
                "attacks": [{"name": attack.technique}],
            }
        )
        print(summary(result))