LLM Vulnerability Research: MCP, Harnesses, Gates

Most advice about using LLMs for security research is about prompts and model choice. Two long write-ups by security researcher Andy Gill, who publishes as ZephrFish, argue the real gains come from the structure around the model: the tools it can call, the stages the work is split into, and the checks every finding must pass before anyone believes it. This guide summarises both posts and turns them into controls for security teams. Every claim about his systems is his, attributed to him. We have not run his tools or the harnesses he describes.

Step 1: wrap tools as MCP servers

Gill says the first problem he wanted to solve was wrangling: connect to a VM, stage a binary, decompile it, map the attack surface, set up fuzzing, triage crashes, write a proof of concept, draft a disclosure. Each step has its own tool and output format. His answer was to wrap every tool as an MCP (Model Context Protocol) server so the model can call it as a typed function. He reports 8 MCP servers across 5 VMs and over 300 tools, running as separate Python processes under Claude Code and registered in one .mcp.json file.

Step 2: treat every finding as a hallucination until it is proven

Gill organises work into campaigns: a directory under hunts/campaigns/ with a structured layout for findings, crashes, coverage data, notes and disclosures, created with a tool_campaign_create tool. He calls the following the most important design decision in the system, one he wishes he had made on day one: findings start as hallucinations. Every new finding goes into a hallucinations/ folder, not findings/, and is promoted only after passing four gates.

A finding that passes all four is promoted with a tool_finding_promote call. If it is invalidated later, a tool_finding_demote call moves it back. Gill says this has saved him from submitting nonsense to vendors, and that human review stays in the loop so he does not flood programmes with reports. His example: in one early hunt session the engine flagged 6 findings from manual analysis, all 6 were disproven in dynamic analysis, and the hallucinations folder caught every one before the final report.

Step 3: add memory, so each hunt starts smarter

Gill describes a knowledge loop built on a retrieval-augmented generation (RAG) index. In his words, each fuzzing start tool queries the index for prior findings before it launches, and each crash and finding records its outcome afterwards. His five steps:

1. Hunt: before a campaign, ask the knowledge base whether crashes in this binary or application type have been seen, which techniques worked and which defences blocked them.
2. Collect: record every crash with its domain, target and crash class.
3. Enrich: feed in vendor advisories, patch diffs, variant-analysis matches, bounty submissions and coverage plateau events.
4. Learn: index everything automatically in a FAISS vector store with sentence-transformer embeddings.
5. Repeat: the next hunt starts with richer context.

He also keeps a known-defence database: when a target proves to be hardened, that is recorded and lowers the target's priority on future hunts. In his June post he adds that context management decides what a stage sees in one run, while RAG decides what the harness can reuse from previous runs.

Step 4: the harness decides what runs, when and with what

Gill defines a harness as the orchestration layer around an LLM that controls inputs, tools, prompts, models, state, validation gates and outputs for each stage. He says MCP servers sit inside it. In his kit the workflow is deliberately simple: recon, hunt, validate, trace, report.

• Recon maps the target.
• Hunt investigates focused hypotheses, with workers running narrow tasks in parallel under context budgets.
• Validate looks for reasons a finding is wrong.
• Trace proves whether attacker-controlled input reaches the vulnerable sink.
• Report receives only findings that passed those gates.

Stages exchange structured artefacts, not one long conversation, which he says makes the pipeline easier to inspect, rerun and replace. He also describes model routing: cheaper models classify, organise and summarise, and stronger models are kept for validation, tracing and synthesis. He warns against one system prompt for the whole pipeline, and treats the context window as a budget: reduce fuzzer and scanner output to the few hundred tokens that matter before it reaches a prompt.

The harnesses Gill compares

These descriptions are Gill's, from using them. We have not tested any of them.

He also released a stripped-back template, harness-kit, with separate stages, structured artefacts, scoped context, validation gates, model routing and persistent state, and says it is deliberately a template, not a finished autonomous research platform. For validation he points to a tool called Scrutineer, whose revalidate skill checks a high or critical finding against git history and returns true positive, false positive, already fixed or uncertain, so that only likely-real findings reach costly verification.

What security leaders should do this week

• Adopt the gate idea for any AI-found finding. Require a reproducible proof, in a clean environment, before it is reported or ticketed. Label anything else a lead.
• Test as a standard user. Gill's Gate 3 is a good rule: a bug that only works as SYSTEM or admin ranks differently from one that works as an ordinary user.
• Decide who may run an AI harness and where. Use an isolated environment, hard scope and egress controls. See what happened when a researcher's agent left its sandbox.
• Keep humans in the loop before reports leave. The cost of a flood of unverified reports falls on vendors, and on your reputation.
• Treat MCP servers as privileged. They give a model real tools. Review each one before connecting it, for example with MCP Guard.
• Log what the agent did. Structured artefacts per stage give you an audit trail, which is what Gill says makes the pipeline easier to inspect and rerun.

Put it to work

• AI Pentesting Detector Lab: spot an automated, AI-assisted probe in your logs.
• Agentic AI Security Lab: look at the permissions and tool access that make agents risky.
• AI Red Teaming Lab: practise a red-team exercise against an AI system.
• Vulnerability Prioritizer: rank what the harness finds against your own exposure.

Related news: 0 by 0.security, an open-source AI security harness.

FAQ

What is an LLM harness in vulnerability research?

Andy Gill defines it as the orchestration layer around an LLM that controls the inputs, tools, prompts, models, state, validation gates and outputs for each stage of work. MCP servers sit inside it as one kind of tool: they give the model callable functions, but the harness decides when, in what order and with what context they are called.

What is the difference between an MCP and a harness?

Per Gill, an MCP server exposes tools to the model as typed functions, such as decompiling a function or running a command. A harness coordinates them: it picks the stage, the model and the context, and decides what to do with the result. He says you can have a full set of MCP servers and still get inconsistent, unverifiable output if nothing coordinates them.

How do you stop an AI vulnerability hunter reporting false findings?

Gill's pipeline treats every new finding as a hallucination until it passes four gates: a proof of concept exists and compiles; it reproduces the crash in a clean VM snapshot; the crash is exploitable and not just a null dereference or graceful exit; and the bug triggers as a standard user, not SYSTEM or admin. He reports that in one early session all six findings the engine flagged were disproven in dynamic analysis, and the gate caught every one before the report.

Which open-source LLM harnesses does the author recommend?

Gill says RAPTOR is the one he uses most. He also describes Anthropic's defending-code-reference-harness, Baby Naptime, evilsocket's audit framework, Visa's vulnerability agentic harness and his own harness-kit template. His descriptions of them are his opinion from using them; we have not tested any of them.

Does an AI-found vulnerability count as confirmed?

Not until it is verified. Gill says of Visa's harness that it treats results as triage candidates rather than confirmed vulnerabilities, and that LLM-led source pipelines still need human review. Treat model-found leads as leads until a reproducible proof exists and a human has reviewed it.

What should a security team do before running an AI harness?

Decide who may run one and against what; run it in an isolated environment with egress controls and a hard scope; require reproduction evidence before any report leaves the team; and keep a person in the loop so you do not flood vendors with unverified reports, which Gill says he designed his pipeline to avoid.

Sources

• Andy Gill (ZephrFish): Jenny was a Friend of Mine - MCPs and Friends (4 Apr 2026)
• Andy Gill (ZephrFish): Harnessing Harnesses - Climbing the LLM Hills (27 Jun 2026)

This article is educational guidance for security leaders, not legal advice. Only use these techniques against systems you own or are authorised to test.