Agent Plugin & Skill Supply-Chain Lab
An AI agent is about to add third-party components to its tool loop — an unvetted MCP server from a random registry, a popular but unpinned skill that auto-updates, a pluginasking for far more scope than its job needs, a typosquatted dependency name, a script piped from a gist, and — importantly — a signed, pinned component from a publisher you already trust. For each, decide the component type, the supply-chain risk, and the control that actually fits — because an unvetted source, an over-broad scope and a silent auto-update each call for a different control, and a properly vetted component should be added, not blocked. Grounded in Google Research's Harness² (arXiv 2610.14224); generic examples only, no invented CVEs.
The real job. More noise, a tighter clock.
Component intake queue
1 / 9 triaged
anonymous publisher on a random third-party MCP registry
surface · file read/write + outbound HTTP tools exposed to the agent
The agent needs to parse PDFs; this server turned up first in a web search and advertises a one-line install.
1 · Component type
2 · Supply-chain risk
3 · Control / safe-to-add decision
Pick a component type, the supply-chain risk, and a control to continue.
Risk × scope map
This component: decide its risk and how broad its scope is.
Triage
Third-party components an agent is about to add. For each of 9, decide the component type, the supply-chain risk (or none), and the control that actually fits the risk.
So far: 0 / 9 components triaged.
Live arena
4 activeadversary staging…
What this lab is
- A free, browser-based triage lab: an agent is about to add a stream of third-party components — plugins, skills, MCP servers, scripts and code dependencies — and you decide the supply-chain risk on each before it enters the tool loop.
- For every component you pick the COMPONENT TYPE, the SUPPLY-CHAIN RISK (unvetted source, malicious/typosquat, over-permissioned, unpinned auto-update, or none/vetted), and the CONTROL that actually fits.
- It is grounded in Google Research's Harness² (arXiv 2610.14224), which treats plugins, skills and scripts as editable harness components in the agent's tool loop — so every added component is attack surface — and in standard software/AI supply-chain controls (SLSA-style provenance, version pinning, MCP vetting) and the ISO/IEC 27090 supply-chain threat framing.
- The core skill it drills: match the control to the risk — vet and sandbox unvetted sources, pin versions and gate upgrades for auto-updating components, scope to least privilege for over-permissioned ones, reject typosquats — and do NOT over-block a signed, pinned, provenance-verified component that is safe to add.
- A component intake queue plus a live risk × scope map, difficulty tiers, a speed bonus and an S–D grade.
The control follows the risk
Harness² (Google Research, arXiv 2610.14224) makes a point that reframes agent security: the plugins, skills and scripts in an agent's harness are editable components inside the tool loop, not fixed infrastructure. That means every third-party component the agent adds is live code with access inside the loop — and therefore part of the supply chain you have to vet. The queue in this lab is built to force the distinctions that matter. An unvetted source — an anonymous MCP server from a random registry, a script from a paste site — needs trusted-registry sourcing, code and publisher review, and sandboxing; popularity in a search result is not vetting. An unpinned auto-update — a skill on @latest, a dependency on a floating range — needs an exact pin and a reviewed upgrade step, because the source being trusted does not stop a future (or compromised) release from flowing in unreviewed. An over-permissioned component — even one from a trusted vendor — needs least-privilege scope, because the gap between what it does and what it can do is the attack surface. And a typosquat gets rejected outright: a one-character-off name is a dependency-confusion attack, and install-time code runs before anything "crashes."
Just as important: do not over-block. A component that is signed, its build provenance attested (SLSA-style), pinned by exact version or hash, and scoped to least privilege from a known publisher is the vetted case — the right call is to add it and keep monitoring, not to reject it on principle or to loosen the very controls (pinning, provenance) that make it safe. The whole skill of supply-chain triage is telling the over-trusted, over-scoped, unpinned component apart from the one that is already doing the right things, and spending your review effort where it moves risk the most.
Frequently asked questions
What does the Agent Plugin & Skill Supply-Chain Lab teach?
It teaches how to triage the third-party components an AI agent pulls into its tool loop — plugins, skills, MCP servers, scripts and code dependencies — for supply-chain risk before they run. For each component you identify its type, the supply-chain risk it carries (unvetted source, malicious or typosquatted name, over-permissioned scope, unpinned auto-update, or none because it is signed, pinned and vetted), and the control that fits. The lab makes the key move explicit: the control has to match the risk, and the most common mistakes are over-trusting (adding an unvetted or over-scoped component) and over-blocking (rejecting a properly vetted one).
What is Harness² and how does it relate to this?
Harness² is research from Google Research (arXiv 2610.14224) that treats the agent's harness — the plugins, skills and scripts inside its tool loop — as editable components rather than fixed infrastructure. The implication for security is direct: every third-party component an agent adds is live code with access inside the loop, so it is attack surface and belongs in a supply-chain review. This lab drills exactly that review for the component types an agent commonly acquires.
Why does a trusted registry not make an unpinned component safe?
Because the source and the version are two different controls. Pulling from a trusted registry addresses provenance — you know where the component came from. But an unpinned reference (a floating range, or a rolling :latest tag) means the actual running code can change on the next install: a new release, or a maliciously published one from a compromised maintainer account, flows in with no review. Pinning to an exact version or digest and gating upgrades behind a review step is what closes the auto-update risk. You generally need both: a trusted source and a pinned version.
Is this a real scanner, and does it use real CVEs?
No. It is an educational, offline triage exercise with a synthetic queue of components. It does not connect to, install, or scan anything, and it contains no exploit detail — only the defender decision of component type, supply-chain risk and control. The examples are deliberately generic: no real CVE numbers are quoted or invented, and no named vendor is accused of anything. The controls mirror established practice — SLSA-style build provenance and signature verification, version pinning, least-privilege scoping, sandboxing and trusted-registry vetting, consistent with the supply-chain threat framing in ISO/IEC 27090.