Policy Gateway Lab
Between an autonomous agent and the world sits a runtime policy gateway — the enforcement layer that runs a fixed set of checks on every action the agent takes. A tool call, a request, a drafted reply, a network send arrives; the gateway must allow it, block it, or escalate it to a person. Some actions hide a prompt injection, a leaked secret, data heading to an external host, or a runaway loop. Others are a customer simply reading their own order — and blocking those is its own failure. For each, call the verdict and name the check that owns it: input classifier, output classifier, deterministic rule, egress allow-list or a human gate. Stop the attacks; let the benign traffic through. Defensive only; synthetic scenarios, no working exploits.
The real job. More noise, a tighter clock.
Action queue
1 / 10 ruled on
A read-only lookup of data the caller already owns.
action → call get_order_status(order_id) for the authenticated caller's own order
Runtime policy gateway · checks in order
Tip: inspect the payload before you rule — the action summary alone rarely tells you which check owns it.
1 · Gateway verdict
2 · Responsible check
Pick a verdict and the responsible check to continue.
Impact × verdict map
This action’s potential impact is 1/5. Decide how far the gateway should let it through — allow, escalate, or block — and which check owns that call.
Guardrail shift
A queue of 10 agent actions arriving at the runtime policy gateway. For each, call the verdict — allow, block, or escalate — and name the responsible check. Some are benign and in-scope; do not over-block.
So far: 0 / 10 actions ruled on.
Live arena
4 activeadversary staging…
What this lab is
- A free, browser-based guardrail drill: play the runtime policy gateway in front of an autonomous agent. For each action that arrives, rule ALLOW, BLOCK, or ESCALATE-to-human, and name which check is responsible.
- Five real guardrail checks plus a clean pass: input classifier, output classifier, deterministic rule, egress/destination allow-list, human-in-the-loop gate — or no check trips (benign, in-scope traffic).
- The attacks must be stopped at the right check — an injected request at the input classifier, a leaked secret at the output classifier, an off-allow-list POST at the egress boundary, a runaway loop at the deterministic cap. Irreversible and genuinely ambiguous actions get a human.
- Several actions are benign on purpose — the caller reading their own order, a scoped ticket update, a masked card shown to its owner, a templated email to an allow-listed recipient — so the drill grades over-blocking as hard as under-blocking.
- Grounded in the OWASP Top 10 for LLM Applications and the OWASP Agentic initiative (Excessive Agency, Prompt Injection, Sensitive Information Disclosure) and NIST AI RMF’s MANAGE function. Synthetic, defensive-only scenarios; no working exploits. Difficulty tiers, a speed bonus and an S–D grade.
The right check, at the right place in the pipeline
A guardrail layer is defence-in-depth: no single check carries the whole load, and each class of risk has a place it is cheapest and most reliable to catch. A direct injection or a jailbroken request belongs to the input classifier, which also screens untrusted tool and RAG content so an indirect injection riding in on a fetched page is treated as data, not an instruction. A secret or another customer’s PII in the response is the output classifier. Data leaving to an unapproved host is the egress allow-list at the network boundary — and it only works if it matches the registrable domain exactly, never a substring. A bulk irreversible delete or a blown spend budget is a deterministic rule, because a fixed ceiling fires the same way every time where a model’s judgment does not.
Just as important: do not over-enforce. A gateway that escalates every email and blocks every write trains its humans to rubber-stamp and makes the agent useless — so the benign cases here are scored exactly as hard as the attacks. An authenticated read of the caller’s own record, a reversible in-queue status change, a masked card shown to its owner, a templated message to an allow-listed recipient: those are ALLOW, and nothing in the pipeline should trip. The human-in-the-loop gate is reserved for the genuinely high-impact, ambiguous call — the company-wide announcement, the production rollback mid-incident — not the routine action that merely looks scary. Match the verdict and the check to where the risk actually lives.
Frequently asked questions
What does the Policy Gateway lab teach?
It trains the decision a runtime guardrail layer makes on every agent action: should the gateway allow it, block it, or route it to a human — and which of its checks is responsible for that call? You work a queue of agent actions (a tool call, a request, a drafted response, a network send) and for each you pick the verdict and the responsible check from the gateway’s fixed set: input classifier, output classifier, deterministic rule, egress allow-list, human-in-the-loop gate, or none. It is runtime-enforcement practice; it does not inspect any real system.
Are these real guardrail checks?
Yes — the five checks are the standard building blocks of a defence-in-depth guardrail / runtime-enforcement layer: moderation classifiers on the model’s input and output, deterministic (non-model) policy rules such as irreversible-op patterns and spend/iteration caps, an egress or destination allow-list at the network boundary, and a human-in-the-loop approval gate for high-impact actions. The scenarios map to named risks from the OWASP Top 10 for LLM Applications and the OWASP Agentic Security initiative (Prompt Injection, Sensitive Information Disclosure, Excessive Agency, Unbounded Consumption) and to NIST AI RMF’s MANAGE function. The individual scenarios are synthetic, illustrative examples, not captured data.
Does it attack an agent or generate exploits?
No. This is strictly a defensive training drill. Every action is a synthetic, illustrative example and every payload is defanged — injected instructions are quoted as text, external hosts are written as evil-sink[.]io, and secrets are shown as redacted placeholders. Nothing here can be run against a live agent or gateway; it only trains the judgment of where in a guardrail pipeline each class of risk should be caught.
Why does it grade over-blocking as a failure?
Because a gateway that blocks real attacks but also blocks the customer reading their own order gets switched off — and then nothing is protected. Several scenarios are deliberately benign and in-scope: an authenticated read, a reversible scoped write, a masked card shown to its owner, a templated email to an allow-listed recipient. Calling one of those BLOCK or ESCALATE is scored as wrong, because least-friction enforcement means the RIGHT verdict, not the most restrictive one. Educational simulation, not security advice for a specific system.
This tool is free and always will be. Would a Pro workspace — saved designs, team sharing and exports — be worth paying for?
Show me Pro →No account needed to use this tool — this just tells us whether Pro is worth building out.