Insecure Output Lab
A queue of flows lands on your desk, each one an LLM or agent output heading into a sink — one is rendered as HTML, one is concatenated into a SQL query, one is passed to a shell, one becomes a URL the server fetches, one a file path, one is handed to eval. For each, call the vulnerability class it creates, the principle at stake, and the mitigation that closes it at the sink — because model output is attacker-influenceable, so the sink must treat it like any untrusted input: encode it, parameterize it, never execute it, allow-list what it can reach. A few flows are already handled correctly — don’t over-block those. Real OWASP and CWE terminology, a safe simulation, no working exploits.
The real job. More noise, a tighter clock.
Output-flow queue
1 / 10 reviewed
Feature: Show the assistant's formatted answer in the support chat UI.
Data flow (defanged pseudocode)
element.innerHTML = modelOutput // model answer inserted as raw HTML, then persisted and re-shown to other users
1 · Vulnerability class
2 · Secure-output principle
3 · Mitigation that makes it safe
Pick the vulnerability class, the principle, and a mitigation to continue.
Severity × blast-radius map
This sink: decide what class an unhandled output creates here, and how to close it.
Output handling
A queue of flows where LLM/agent output reaches a sink. For each of 10, name the vulnerability class, the principle at stake, and the mitigation that closes it at the sink. Some flows are already safe — do not over-block.
So far: 0 / 10 flows reviewed.
Live arena
4 activeadversary staging…
What this lab is
- A free, browser-based lab: work a queue of flows where an LLM/agent OUTPUT lands in a downstream sink — rendered as HTML, built into a SQL or NoSQL query, passed to a shell, used as a URL the server fetches, used as a file path, or executed as code — and decide what goes wrong and how to stop it.
- For each flow you pick the VULNERABILITY CLASS (XSS, SQL/query injection, OS command injection, SSRF, path traversal, code injection / unsafe eval, or none), the secure-output PRINCIPLE at stake, and the MITIGATION that closes the class at the sink.
- Grounded in OWASP "LLM02: Insecure Output Handling", MITRE ATLAS, and the real CWE sink classes (CWE-79, 89, 78, 918, 22, 94, 502) with real control names — contextual output encoding, DOMPurify, Content-Security-Policy, parameterized queries, argv arrays, egress allow-lists, path canonicalization and data-only deserializers.
- The core skill it drills: treat model output as untrusted input to the sink — encode for the context, parameterize instead of concatenating, never execute output, and allow-list the paths and hosts it can reach — while NOT over-blocking a flow that is already correctly encoded, bound or confined.
- A safe educational simulation: every flow is defanged pseudocode, nothing executes, no working exploits. Difficulty tiers, a speed bonus and an S–D grade.
The vulnerability lives at the sink, not in the model
Insecure Output Handling is the quiet counterpart to prompt injection. Injection decides what the model says; insecure output handling is what your code does with what it said. The same string is harmless in one place and catastrophic in another: assigned to textContent it is inert text, assigned to innerHTML it is stored XSS; bound as a query parameter it is data, concatenated into SQL it is injection; printed to a log it is noise, passed to a shell or eval it is remote code execution. So the control always belongs at the sink: contextual output encoding or allow-list sanitization plus a Content-Security-Policy for the DOM, parameterized queries for databases, an argument array and no shell for commands, an egress allow-list for outbound fetches, canonicalization and a base-directory check for paths, and a data-only loader for deserialization.
The second discipline is not to over-correct. Keyword blocklists (“strip the word script”, “remove ../”) feel like fixes and are reliably bypassed, while a flow that already uses the correct control — an escaped display, a bound parameter, a canonicalized and allow-listed path — needs nothing added. Bolting redundant filters onto a safe flow buys no security and breaks valid data. The whole skill of output-handling review is telling the dangerous sink apart from the already-safe one, and spending your encoding where the execution actually happens. Grounded in OWASP’s “LLM02: Insecure Output Handling”, MITRE ATLAS and the underlying CWE sink classes.
Frequently asked questions
What does the Insecure Output Lab teach?
It teaches Insecure Output Handling — the OWASP Top 10 for LLM Applications risk ("LLM02") where a downstream component trusts LLM or agent output and passes it, unvalidated and un-encoded, into a sensitive sink. Each scenario shows a flow (model output going into the browser DOM, a SQL/NoSQL query, a shell, a server-side fetch, a file path or a code interpreter) and you decide the vulnerability class it creates (XSS, SQL/query injection, OS command injection, SSRF, path traversal, code injection, or none), the secure-output principle at stake, and the concrete mitigation that closes the class at the sink.
Is it safe and legal — does it run real attacks or ship exploits?
Yes, it is safe, and no. This is a safe educational simulation, like a tabletop exercise — the same spirit as our other labs. Every data-flow is defanged pseudocode that describes an unsafe path; nothing executes, there are no working exploit payloads, no real secrets, hosts or sinks, and no copy-pasteable weaponized code. It is purely a defensive recognition drill — you learn the attacker's move so you can block it — and using it is entirely legal and educational.
Why is "it came from our own model, so it's safe" wrong?
Because model and agent output is attacker-influenceable: a crafted prompt, an indirect injection from a tool result or a poisoned RAG document can steer what the model emits. So output must be treated as untrusted input to whatever sink consumes it — exactly like user input. Rendering it as raw HTML is XSS; concatenating it into a query is injection; passing it to a shell or eval is code execution; using it as a URL or path is SSRF or traversal. The source being "internal" changes nothing about the sink's duty to validate and encode.
Can you over-block output handling?
Yes, and the lab tests for it. Output that is set via textContent or auto-escaped, bound as a query parameter and returned as JSON, or used as a path that is canonicalized and confined to an allow-listed base directory, is already safe — the correct control is in place. Reflexively adding keyword filters, banning characters or removing the feature buys no security and breaks legitimate behaviour. The skill is matching the mitigation to the sink: closing the real class where it exists, and recognising the already-safe flow where it does not.