Crescendo Attack Lab: Defend Against Multi-Turn Jailbreaks
This is a hands-on lab about catching a slow, multi-turn attack on a chat assistant. A crescendo conversation never sends one bad message; it escalates a little each turn, using the assistant’s own replies. You review a fictional support chat and make 12 scored calls on conversation-level scoring, what the checker sees, limits, handoff and keeping secrets out of the prompt.
It follows the published Crescendo research and Microsoft’s write-up, described at the level they describe it. The target in the story is harmless and no attack text appears. For the single-message side see the Prompt Injection Tester.
You are Priya Raman, security lead at Harbourline Telecom. The customer-support assistant Pip answers billing and plan questions. A researcher paper describes "crescendo" attacks: a conversation that starts with an ordinary question and escalates a little each turn, using the assistant's own replies, so that no single message looks wrong. In the lab's fictional case the escalation aims at something harmless to read about: coaxing Pip into revealing its internal discount rules. Over 12 scored decisions you review a transcript turn by turn and choose detection and control. Nothing here is a working attack and no jailbreak text appears. Each call scores 0 to 10 with the source behind the feedback.
Make it yours
Replace the sample names so the scenario and the debrief you export are about your organisation.
TL;DRWhat this is
5 points
- A scenario lab about the defender's side of multi-turn "crescendo" attacks: a conversation that starts with an ordinary question and escalates a little each turn, so no single message looks wrong. You act as security lead at a fictional telecom whose support assistant is being coaxed into revealing its internal discount rules.
- You make 12 scored decisions: why single-message filters miss it, how to treat the early turns, which trajectory signals to score, per-turn versus cumulative checks, designing the conversation risk score, full history versus summaries, the response ladder, turn and rate limits, moving secrets out of the prompt, logging, the human handoff, testing and what you tell leadership.
- It draws on the Crescendo paper by Russinovich, Salem and Eldan, Microsoft's own write-up of how it mitigates the attack, and OWASP guidance on prompt injection and sensitive information disclosure. Claims are attributed ("the authors report", "Microsoft says") and we quote no success rates.
- Free to play, no account needed. The escalation topic is deliberately harmless, no jailbreak text appears and every name is invented.
- Defensive and educational only.
How this differs from our other agent labs
The Prompt Injection Tester builds single probes to run against your own app; this lab is about sequences, where every message passes. The AI Prompt Injection Incident Lab is an incident response story about a leak through a ticket, and the Agent Prompt Injection Harness Lab triages injection vectors against agents. Here the attacker is a user talking to the assistant directly, slowly.
Content planted in files, memory or tools is covered by the Poisoned Instruction File Lab, the Agent Memory Governance Lab and the MCP Rug Pull Lab. For untrusted content passing between agents, try the Inter-Agent Sanitization Lab.
Frequently asked questions
What is a crescendo attack on an LLM?
It is a multi-turn way of steering a language model. The authors of the Crescendo paper describe it as starting with a general, seemingly benign question and then gradually escalating the dialogue by referring to the model's own replies. Because each message is a small step from the last, a check that reads one message at a time sees nothing wrong. This lab covers the defender's side and uses a harmless target, an assistant's internal discount rules.
Why do single-message filters miss multi-turn attacks?
Microsoft says that each individual prompt is not on its own a threat, that keywords alone are insufficient, and that only the combination is clear. It adds that the model itself sees nothing out of the ordinary because every step is rooted in something it said earlier. The fix is to assess the conversation as well as the message.
Does looking at the whole conversation help?
Microsoft says that passing a larger context window to its existing malicious-intent detectors, without improving the detectors, significantly reduced the efficacy of Crescendo. That is a vendor statement and we do not quote a figure, but it supports adding a conversation-level check next to the per-turn one. It does not remove the risk.
Is a turn limit enough to stop crescendo attacks?
No. Microsoft says these attacks can usually be achieved in fewer than 10 interaction turns, so a normal-looking limit will not stop one. Limits and rate limits are still useful layers: they bound cost, slow probing and make it visible. The lab teaches limits as one layer among several.
What should an assistant never keep in its prompt?
Anything whose disclosure would matter. OWASP warns that restrictions in a system prompt may not be honoured and can be bypassed by prompt injection, and advises handling privileged functions in code rather than giving them to the model. In the lab you move the discount rules into a backend the assistant calls, so a leaked prompt is embarrassing rather than costly.
Does this lab teach me to jailbreak models?
No. It describes the attack only at the level the published paper and Microsoft's write-up do. The escalation topic is benign, turns are described by intent rather than quoted, and no harmful-content or jailbreak text appears. Every organisation and assistant is fictional.
How is the lab scored?
Each of the 12 decisions scores 0 to 10 and the result is your share of the maximum, with a short band that says what the score means. The debrief shows the source behind each rationale. Lines labelled PlayCISO practice are our reasoning, not a statement by a source.
Sources
All read on 4 November 2026. Lines in the feedback marked “PlayCISO practice” are our own reasoning, not a statement by any source. The scenarios are fictional. We cite primary sources, not vendor advisories, and say where a page could not be fully read.
- Russinovich, Salem and Eldan (arXiv 2404.01833): Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack (Submitted 2 April 2024, version 3 dated 26 February 2025) We read the abstract page. The authors describe a multi-turn attack that starts with a general question and escalates by referring to the model's own replies. We quote none of their success rates and reproduce no attack prompts.
- Microsoft Security Blog: How Microsoft discovers and mitigates evolving attacks against AI guardrails (11 April 2024 (from the URL)) Microsoft's own write-up. It says single prompts look harmless on their own and that passing a larger conversation context to existing intent detectors significantly reduced Crescendo's efficacy. Vendor statement, not independent testing.
- OWASP GenAI Security Project: LLM01:2025 Prompt Injection (2025 edition) Says it is unclear whether fool-proof prevention exists; lists mitigations including output validation, least privilege, human approval for high-risk actions and segregating external content.
- OWASP GenAI Security Project: LLM02:2025 Sensitive Information Disclosure (2025 edition) Gives general least-privilege guidance and lists security credentials as one kind of sensitive information; it has no credential-specific guidance.
More security tools
The tools people use most on PlayCISO.
Catch injected instructions hiding in content your AI agent should only read.
Score your readiness against the first international standard for AI security threats.
Critical CVEs across 21 major vendors, month by month — and what the 2026 surge means.
Build an ordered ruleset, replay labelled packets and catch shadowed and any-to-any rules.
Self-assess all six Functions, from Govern to Recover, and see your weakest one.
Search 172 threat actors across Microsoft, CrowdStrike, Mandiant and community names.