AI Pentesting Detector: Is It an Agent or a Human?
This lab trains you to spot AI pentesting in logs. You are the SOC lead for a fictional company. Three intrusions arrive with synthetic log excerpts, and for each you decide whether it looks agent-driven, human-driven or a legitimate authorised AI pentest, then choose what to hunt, alert on and contain.
The lesson: an autonomous agent is not stealthier, it is noisier, and it defeats hunts built for a human. It pairs with the Agent Defense Evasion Hunt and the Detection Lab. Everything is invented and no exploit code appears.
You are Dana Whitlock, SOC lead at Northgate Telecom. Three fictional intrusions land on your desk in one week. Each has synthetic log excerpts: auth failures, scan bursts, lateral hops and command timing. For each you decide whether it looks agent-driven, human-driven or a legitimate authorised AI pentest, using cadence, parallelism, 24-hour rhythm, error handling and tool fingerprints. Then you choose what to hunt, alert on and contain. The lesson, in line with what Malbear Labs reports in its Operation Snowball write-up: an AI agent is not stealthier, it is noisier and more repetitive, and it defeats hunts that look for a human (working hours, interactive-session timing). All data is invented, uses documentation-reserved addresses and contains no exploit code.
This is the guided version: a yellow flash shows where to click and a bar says what to do. Switch to Free play any time.
GuidedPress the highlighted Start button. You can edit the names above first, or keep the samples.
Make it yours
Replace the sample names so the scenario and the debrief you export are about your organisation.
Loading…
Sample report · illustrative data
Sample result: AI pentesting detector debrief
Example debrief for a SOC lead who classified three synthetic intrusions and chose hunts, alerts and containment. Scores are illustrative.
Result
- Score
- 83 / 120 (69%, example)
- Band
- Solid, with human-shaped blind spots
- Decisions
- 12 scored 0 to 10; best answer is 10
Classification of the three intrusions
| Intrusion | Signals in the logs | Verdict |
|---|---|---|
| A: VPN sweep and scan burst | Fixed 350 ms spacing, 27 accounts, three sources in the same second | Agent-driven (automated, parallel) |
| B: jump host foothold | Flat 24-hour activity for 19 days, quick tool switch after a block | Agent-driven, hostile |
| C: staging web scan | Parallel and regular, but matches an engagement register entry | Legitimate authorised test, confirmed with the contact |
Where points were lost (example)
- Kept an odd-hours interactive-session hunt as the main hunt.
- Contained by blocking each new source address instead of disabling accounts and isolating hosts.
- Treated a user-agent string as the main test.
Takeaway
Treat an agent as noisy and regular, not quiet. Hunt for cadence, breadth, tunnels and watchdog-style files, alert with behavioural endpoint monitoring, and contain on identity and host. Lines marked PlayCISO practice in the debrief are our reasoning.
Made-up example data. Your own run uses your inputs. Downloads of the full report need Pro.
TL;DRWhat this is
5 points
- A scenario lab on AI pentesting and autonomous pentesting from the defender's side: you review synthetic log excerpts from three fictional intrusions and decide which look agent-driven, which human-driven and which are a legitimate authorised AI pentest.
- You make 12 scored decisions on login cadence, scan-burst parallelism, 24-hour rhythm, error handling after a block, tool fingerprints, deconflicting an authorised test, the stealth myth, what to hunt, what to alert on, what to contain and what to tell the board.
- Malbear Labs reports, in its 8 November 2026 Operation Snowball write-up, an autonomous agent with a primary agent delegating to workers, a credential test paced at one request every 0.35 seconds and a fresh relay per session. The post says agent activity is noisier and more repetitive than human activity, and advises against relying on hunting for a human operator. This is one vendor's analysis of a recovered workspace.
- The public READMEs of PentAGI, Strix and PentestGPT describe multi-agent, tool-driven testing and say to use them only on systems you own or are authorised to assess, which is why the lab includes a legitimate-test case.
- Logs are synthetic and use documentation-reserved addresses and example.com. No exploit code, no real indicators, victims or operators. Defensive and educational only.
Frequently asked questions
What is AI pentesting?
AI pentesting is penetration testing in which software agents plan and run steps such as reconnaissance, scanning and validation with little human input. Open-source examples include PentAGI, which describes itself as a multi-agent system for autonomous penetration testing, Strix and PentestGPT. Their READMEs say to use them only on systems you own or are authorised to assess.
How can I tell AI penetration testing from a human attacker in my logs?
Look at several signals together: a fixed or very regular cadence, parallel activity against different targets in the same second, activity round the clock with no human-shaped gaps, quick switching of technique after a block, and tool fingerprints. No single one is proof. Malbear Labs reports agent activity as noisier and more repetitive than human activity. Combining the signals is PlayCISO practice.
Are autonomous pentesting agents stealthier than humans?
Not according to the evidence in the lab's main source. Malbear Labs says agent activity is noisier and more repetitive, and that the problem for defenders is hunts built around a human operator, such as working hours and interactive-session timing. That is one vendor's analysis of one campaign, so treat it as evidence rather than a law.
How do I tell a legitimate AI pentest from an attack?
Match the traffic against a written engagement record: source address, scope, time window and a reachable tester contact, and confirm out-of-band. A header or tool name is only a claim. Authorised tools state that you must have permission, so a real test has an owner and a scope. The deconfliction steps are PlayCISO practice.
Should I block the IP address of an AI hacking tool?
Only as a stop-gap. Malbear Labs reports an agent that kept a local database of more than 4,000 public relay servers and picked a fresh one per session, so address blocks chase it. Contain on identity and host: disable accounts and credentials, isolate hosts and cut tunnels.
Does bot traffic detection work against AI agents?
Partly. Bot traffic detection based on request regularity, breadth and parallelism helps, but user-agent strings are easy to change and many agents drive ordinary tools. Use fingerprints as a supporting signal and rely on behaviour, including endpoint behavioural monitoring.
Sources
Read on 2026-11-08. Lines in the feedback marked “PlayCISO practice” are our own reasoning, not a statement by any source. The scenarios are fictional and use made-up data.
- Malbear Labs: Operation Snowball, inside an autonomous AI intrusion campaign (8 Nov 2026) (the author's reported findings, not independently verified)
- PentAGI README (vxcontrol) (multi-agent autonomous penetration testing; authorised use only)
- Strix README (usestrix) (parallel multi-agent testing; authorised use only)
- PentestGPT README (GreyDGL) (educational and authorised testing only)
More security tools
The tools people use most on PlayCISO.
Catch injected instructions hiding in content your AI agent should only read.
Score your readiness against the first international standard for AI security threats.
Critical CVEs across 21 major vendors, month by month — and what the 2026 surge means.
Build an ordered ruleset, replay labelled packets and catch shadowed and any-to-any rules.
Self-assess all six Functions, from Govern to Recover, and see your weakest one.
Search 172 threat actors across Microsoft, CrowdStrike, Mandiant and community names.