AI Defense Lab
Your AI Security Learning Platform
Learning Modules
Threat Detection Lab
Run attack payloads through the live detection pipeline and read the evidence it produces
Key Concept: Five detectors score the input with weighted patterns, matched against the raw text and normalised variants — instruction override, data exfiltration, encoding evasion, tool hijacking, RAG poisoning. Their findings feed a versioned policy engine, which produces the decision. Measured efficacy is on the dashboard, misses included.
Policy Design Lab
Build rules that decide what to block, allow, or flag for review
Key Concept: Policy = Rules + Actions. Each rule maps detector findings to a decision: ALLOW, BLOCK, or REQUIRE_APPROVAL.
RAG Security Lab
Understand how retrieval-augmented generation can be poisoned and defended
Key Concept: OWASP LLM08 — the retrieval layer is itself the attack surface: a planted document delivers the injection by construction, a vector store without tenant scoping returns one customer's chunks to another, and embeddings can be partially inverted to recover source text. Trust levels are recorded and displayed, but retrieval does not currently weight by them — so they document risk rather than reduce it.
Agent Attack Lab
See how multi-stage AI agents can be attacked at each pipeline stage
Key Concept: Every stage is an injection point, and the taxonomy separates two of them: the task stage is ASI01 Agent Goal Hijack, the tool-call stage is ASI02 Tool Misuse & Exploitation. Tool calls are simulated, never executed.
Unverified source — OWASP Top 10 for Agentic Applications (2026) — Identifiers taken from a secondary summary, pending confirmation against OWASP's own publication.
Command Injection Lab
See how model output becomes the injection payload for whatever runs next
Key Concept: OWASP LLM05 Improper Output Handling, and ASI05 Unexpected Code Execution (RCE) in the agentic list. The model does not need to be compromised — only persuaded to emit a payload the next system executes. This lab shows the payload shape and the detection pass; it simulates tool execution rather than performing it.
Unverified source — OWASP Top 10 for Agentic Applications (2026) — Identifiers taken from a secondary summary, pending confirmation against OWASP's own publication.
Audit Trail Explorer
Every detection creates a traceable, immutable audit trail
Key Concept: Immutable audit trails for accountability. Traces enable replay, debugging, and compliance reporting.
Your Progress
Block rate is the share of your policy-evaluated runs that the policy blocked — RAG retrieval queries are excluded, since no policy judges them. It is not detector accuracy: every run here is one you started, usually on an attack scenario you picked deliberately, and there is no labelled ground truth to measure misses against.
—
Total Runs
—
Runs Blocked
—
Active Policies
—
Block Rate
What these detectors actually catch
Measured against a labelled corpus of 34 attacks and 30 benign inputs, written from the threat model rather than from the detectors’ own pattern lists — so the test and the code cannot agree by construction. Regenerate with pytest tests/test_detection_efficacy.py -s -k report.
Read this as a regression baseline, not a benchmark. The corpus was assembled by the same people who wrote the detectors, so 82% here is not comparable to a published evaluation on an independent adversarial dataset. It answers “has detection got worse?”, not “how would this fare against a real attacker?” For scale, on independent adversarial datasets pattern-based detectors of this kind catch 2–20% of attacks (ACM CODASPY 2025).
By attack class
By whether patterns can reach it
Character-level obfuscation — homoglyphs, zero-width characters, spacing, leetspeak. Folding the input to a canonical form reaches these.
The hostile wording is plain English. A pattern can match it directly, so a miss here is missing coverage.
Paraphrase, other languages, or intent split across turns. No pattern list closes these; they need a model that understands meaning.
OWASP Top 10 for LLM Applications
2025 revision — verified 2026-07-27 against genai.owasp.org. Every identifier below links to its own source page.
“Detectors” below means one or more regex detectors in this lab emit that identifier. It is not a claim that the risk is handled. At deployment-realistic false-positive budgets, widely used open-source prompt-injection detectors catch between 2% and 20% of attacks on independent adversarial datasets. This lab measures 82% on its own 34-case corpus, which is not the same measurement and not comparable: the corpus was written by the same people who wrote the detectors, and the six cases it still misses are precisely the ones a published evaluation is full of — paraphrase, other languages, split payloads. ACM CODASPY 2025 (independent study, not a vendor claim).
| ID | Name | Description | Detectors |
|---|---|---|---|
| LLM01 | Prompt Injection | User input alters the model's behaviour or output in ways the developer did not intend. | override_detector, encoding_detector, poisoning_detector |
| LLM02 | Sensitive Information Disclosure | The model or its surrounding application exposes PII, credentials, business data, or its own training material. | exfil_detector |
| LLM03 | Supply Chain | Compromised models, datasets, adapters, or packages enter the application through its dependencies. | No detector |
| LLM04 | Data and Model Poisoning | Training or fine-tuning data is manipulated to implant bias, backdoors, or degraded behaviour in the model itself. | No detector |
| LLM05 | Improper Output Handling | Model output is passed to a downstream system without validation, turning the LLM into an injection relay. | No detector |
| LLM06 | Excessive Agency | The model is granted more functionality, permission, or autonomy than the task requires. | No detector |
| LLM07 | System Prompt Leakage | The system prompt is disclosed, revealing instructions, logic, or secrets embedded in it. | override_detector |
| LLM08 | Vector and Embedding Weaknesses | The retrieval layer itself is the attack surface — poisoned documents, cross-tenant leakage, or inversion of stored embeddings. | poisoning_detector |
| LLM09 | Misinformation | The model produces confident, plausible, and wrong output that downstream users or systems act on. | No detector |
| LLM10 | Unbounded Consumption | Uncontrolled inference volume or size drives denial of service, cost exhaustion, or model extraction. | No detector |