Agent Prompt-Injection & Harness-Poisoning Lab
Untrusted content keeps reaching your agent — a tool result, a retrieved document or web page, a file, a subagent message, or a persisted memory/“lesson” — and some of it carries hidden instructions. For each scenario, decide where the injection entered, whether the described controls contain it or let it land, and the single defense that actually fits — because treating content as data, scoping tools, mediating output and gating privileged actions each answer a different case. In a self-improving harness (Google Research’s Harness², arXiv 2610.14224) a landed injection can write a durable poisoned lesson, so what you let the harness persist matters most of all. Grounded in the OWASP LLM Top 10 and ISO/IEC 27090 — abstract, defender-side, no payloads.
The real job. More noise, a tighter clock.
Scenario queue
1 / 9 triaged
Hidden directive in a fetched page · Web-fetch tool result
Untrusted content: A page the agent fetched contains, beneath the visible article, a block of text addressed to the AI that tells it to abandon its task and do something else. The directive is described abstractly — the point is only that untrusted page text reached the model.
Controls described: The agent concatenates the whole tool result into its context and has no rule separating instructions from retrieved data, so it starts following the embedded directive.
1 · Injection vector — where the untrusted instruction entered
2 · Outcome — given these controls
3 · Best defense
Pick a vector, the outcome, and the best defense to continue.
Vector × blast-radius map
This scenario: name the vector and judge how far it reaches.
Triage
Untrusted content keeps reaching the agent. For each of 9 scenarios, decide the injection vector, whether it is contained or lands, and the single defense that actually fits.
So far: 0 / 9 scenarios triaged.
Live arena
4 activeadversary staging…
What this lab is
- A free, browser-based triage lab: work a queue of scenarios where untrusted content — a tool result, a retrieved document or web page, a file, a subagent message, or a persisted memory/"lesson" — reaches an AI agent carrying hidden instructions, and decide the risk on each.
- For every scenario you pick the injection VECTOR (where the untrusted instruction entered), whether it is CONTAINED or LANDS given the described controls, and the single best DEFENSE.
- The defender toolkit it drills, straight from the OWASP LLM Top 10: treat tool/retrieved/peer content as data not instructions (LLM01), mediate output before anything downstream renders it (LLM02), scope tools to least privilege and gate privileged actions behind a human (LLM08).
- It covers the risk a self-improving harness adds — Google Research’s Harness² (arXiv 2610.14224): a one-time injection can write a DURABLE poisoned lesson, so you must gate what gets persisted and keep provenance on what is kept. ISO/IEC 27090 frames these AI-specific threats.
- Attacks are described abstractly, defender-side — no working injection strings, no payloads, no invented CVEs — with a scenario queue, a vector × blast-radius map, difficulty tiers, a speed bonus and an S–D grade.
The defense follows the vector
Most agent-security mistakes come from reaching for one control for every finding. The scenarios here are built to force the distinctions that matter. The foundational rule is content as data, not instructions (OWASP LLM01): a tool result, a retrieved document, a file and a subagent message are all untrusted input, and an agent must never execute a directive it finds inside one. But treating input safely is only half the boundary. When the agent emits content into another system, that output needs mediation before anything renders or executes it (LLM02). And because a landed injection can only do what the agent’s tools allow, least-privilege tools and a human gate before privileged or irreversible actions (LLM08, excessive agency) cap the blast radius when prevention fails.
The self-improving harness adds a sharper edge. In Google Research’s Harness² (arXiv 2610.14224), an agent records “lessons” into durable memory and reuses them later — so a one-time injection can become a permanent poisoned lesson. The controls are to gate what the harness is allowed to persist and to keep provenance on any adopted edit, so a poisoned change can be traced and revoked while the legitimate improvements are kept. Just as important: do not over-flag. A scenario where the content-as-data boundary already quarantined the directive is the contained case — the right call is to keep logging and monitoring, not to declare a breach or rip out retrieval. Telling the landed injection apart from the one that went nowhere is the whole skill of triage.
Frequently asked questions
What does the Agent Prompt-Injection & Harness-Poisoning Lab teach?
It teaches how to triage prompt injection against AI agents from the defender’s side. For each scenario you decide where the untrusted instruction entered (the injection vector — a tool result, a retrieved document, a file, a subagent message, or a persisted memory), whether the described controls contain it or let it land, and which single defense actually fits. The core skill is matching the control to the vector and the blast radius: treating untrusted content as data, scoping tools so a landed injection cannot act, mediating output, and — for self-improving harnesses — never persisting an unverified lesson.
Does it contain real injection payloads?
No. Every attack is described abstractly and from the defender’s perspective — the scenario tells you that untrusted content carried hidden instructions and how the agent handled it, never the working string. The lab is a decision exercise about vectors, outcomes and defenses, not a how-to. It cites the standards behind each decision — OWASP LLM01 Prompt Injection, LLM02 Insecure Output Handling, LLM08 Excessive Agency, and ISO/IEC 27090 for AI-specific threats — and invents no CVE numbers.
What is harness poisoning and the Harness² reference?
A self-improving agent harness can write "lessons" into durable memory and reload them on later runs. Google Research’s Harness² work (arXiv 2610.14224) describes how a single successful injection can be captured as a durable poisoned lesson that then steers future, unrelated runs — turning a one-time compromise into a persistent one. The defender controls are to gate what the harness is allowed to persist (don’t write an unverified lesson) and to keep provenance on any adopted edit so a poisoned change can be traced to its source and revoked.
Why does the lab make me pick a defense per scenario instead of one blanket rule?
Because the right lever depends on the vector and the blast radius. Treating content as data (LLM01) stops most injections at the boundary, but a landed injection still needs least-privilege tools and a human gate before privileged actions (LLM08) so it has nowhere to go; output that flows into another system needs mediation (LLM02); and a self-improving harness needs persistence gating and provenance. The lab also includes a case that is already correctly contained — recognising it, and not over-reacting, is part of triage too.