All posts

The LiteLLM Supply Chain Attack: What TeamPCP Stole, Who It Hit, and What to Do Now

supply-chainai-securitylitellmincident-responsethreat-intelligencepypici-cd
August 13, 2026 · PlayCISO

In March 2026, a threat actor tracked as TeamPCP pushed two backdoored versions of LiteLLM — the open-source LLM gateway used by an estimated 95 million monthly PyPI downloads — to the public package index. The malicious releases, versions 1.82.7 and 1.82.8, stayed live for roughly 40 minutes before they were pulled. That was enough. In August 2026, researchers at Hudson Rock and CloudSEK independently obtained the stolen data and confirmed the blast radius: over 2,400 organizations, more than 430,000 CI/CD pipelines, and secrets — cloud keys, SSH keys, AI provider API tokens — sitting in plaintext across a 153GB archive.

This is not a theoretical exposure. If your infrastructure ran LiteLLM 1.82.7 or 1.82.8 at any point in late March 2026, you should treat every credential that touched that environment as compromised until proven otherwise.

How the Attack Actually Worked

TeamPCP never targeted LiteLLM's maintainers directly. They targeted Trivy, the widely used open-source vulnerability scanner, and compromised its software supply chain. LiteLLM's own CI/CD pipeline pulled in the poisoned Trivy build as part of a routine automated scan — and that scan step had access to LiteLLM's PyPI publishing credentials. TeamPCP used those stolen credentials to push two malicious releases straight to the real, official litellm package on PyPI. There was no typosquat, no lookalike package name — this was the genuine package, backdoored at the source.

One of the LiteLLM maintainers has since confirmed the entry point traces directly back to the compromised Trivy scanner running inside their build environment — a reminder that a security tool sitting in your CI pipeline is itself part of your supply chain attack surface.

What the Malicious Payload Actually Did

The backdoored wheels ran a three-stage attack the moment they were installed:

  • Stage 1 — Credential harvesting. Scraped environment variables and config files for OpenAI, Anthropic, Azure, AWS, and GCP keys, SSH keys, and Kubernetes service account tokens.
  • Stage 2 — Lateral movement. Used any stolen Kubernetes credentials to probe for privileged pods and spread inside the cluster.
  • Stage 3 — Persistence. Installed a systemd backdoor service that beaconed out to attacker-controlled infrastructure for further instructions.
Indicators of Compromise
Check for these on any host that ran 1.82.7 / 1.82.8
Type Indicator
Dropped files litellm_init.pth, a tampered litellm/proxy/proxy_server.py
C2 domains models.litellm.cloud, checkmarx.zone
Persistence A systemd service (reported as sysmon.service) beaconing on a schedule
Kubernetes Unrecognised pods named node-setup-* in the kube-system namespace
Behavioural Sudden, unexplained high memory consumption on hosts running the proxy

Scale: What Surfaced in August

The March compromise was quiet — a 40-minute window most teams never noticed. What made it front-page news five months later was the data itself resurfacing. Hudson Rock obtained a 153GB archive containing 433,909 files tied to the TeamPCP campaign, including 118,829 CI/CD runner dumps they were able to attribute to 2,488 distinct corporate domains. CloudSEK, working from a separate dataset of roughly 434,000 stolen files, arrived at a similar figure — close to 2,500 exposed organizations.

Both firms have published domain-checking portals so organizations can look up whether their infrastructure appears in the leaked material. Analyses of the archive have circumstantially linked large, recognizable names — including Nvidia, AWS, Samsung, Intel, Cisco, Salesforce, Boeing, and others — to exposed CI runner dumps. It's worth being precise about what that means: researchers describe this as high-confidence exposure, not confirmed compromise of those companies' production systems. A leaked CI/CD runner dump attributed to a corporate domain proves a credential or environment variable from that environment was in the archive — it does not by itself prove what an attacker did with it. Treat named-company headlines as a signal of scale, not a verified breach report for each name.

What to Do Right Now

1. Check your version history
Check whether 1.82.7 or 1.82.8 was ever resolved in a lockfile, container image, or CI cache — not just what's installed today. The 40-minute window means most affected teams pulled it via automated dependency updates, not a manual install.
2. Pin to a clean version
pip install litellm==1.82.6 (or the current patched release) and pin it explicitly. Purge pip and CI build caches — the malicious wheel can persist in a cache after the lockfile is fixed.
3. Rotate everything, assume burn
Any secret present in an environment that ran the backdoored build — LLM provider keys, cloud credentials, Kubernetes service account tokens, SSH keys — should be rotated. Don't try to determine which specific keys were exfiltrated; assume all of them were.
4. Hunt for the IOCs
Search hosts and clusters for the dropped files, the systemd service, and the node-setup-* pods above. Check egress logs for connections to the listed C2 domains going back to late March 2026.
5. Check the leaked archives
Both Hudson Rock and CloudSEK have published domain-lookup portals against the leaked datasets. If your domain appears, treat it as a confirmed lead requiring full incident response, not a false-positive to dismiss.
6. Audit your CI security tooling
Trivy was the entry point, not the target. Any scanner, linter, or automated tool with write access to your CI pipeline and read access to publishing credentials is a supply-chain risk in its own right. Pin versions, verify checksums, and don't auto-update security tooling inside privileged build steps.

The Pattern Behind This

This is not TeamPCP's first move — the same group has been linked to earlier compromises of both Trivy and the KICS static analysis tool, using one open-source security tool's supply chain to reach into the next project's. That's the part worth sitting with: the attack didn't need to fool a single LiteLLM engineer. It needed to compromise one dependency that every CI/CD pipeline trusts by default because its entire purpose is finding vulnerabilities, not being one.

The lesson generalizes past LiteLLM. Any tool with a privileged spot in your build pipeline — scanners, linters, formatters, test runners — deserves the same scrutiny as a production dependency, because a compromise there inherits every credential the pipeline can see.

Check your own exposure. We built a free, local, single-file LiteLLM / TeamPCP exposure checker — zero network calls, read the source before you run it. And to practice the incident response itself, PlayCISO's War Room runs scenarios built around real compromises — CI/CD breaches, credential leaks, and disclosure timelines under regulatory pressure.

Ready to practise the decisions these articles describe?

Run a free War Room →