AIBOM in Incident Response: Which Model Ran?

When an AI component is compromised or an advisory lands, the AIBOM answers what each release contains, and the deploy log answers which release ran where and when. Together they tell you which model was running, what it could reach and what data it saw. Below is a worked incident on a real class of issue, malicious model files that run code on load, played out on a fictional company.

Who this is for: SOC analysts and incident responders who get the call, and the AppSec engineers who own the model pipeline and will be asked for the evidence.

The issue: model files that run code when loaded

A model saved in a pickle-based format (a PyTorch .bin or .pt file) can execute code when it is unpickled. This is not theoretical. In 2024 JFrog reported about 100 PyTorch and TensorFlow Keras models on Hugging Face with malicious functionality; the example it analysed ran its payload through pickle when the file was loaded. The safetensors format stores tensors only and does not run code on load.

Inference servers have had advisories on exactly this path:

• CVE-2025-24357 (GHSA-rh4j-5rhw-hr54, published 27 January 2025): vLLM called torch.load with weights_only left at False, so a malicious checkpoint could run code. Fixed in vLLM 0.7.0. The advisory notes that safetensors models are not vulnerable.
• GHSA-ggpf-24jw-3fcw (published 22 April 2025): the 0.7.0 fix relied on weights_only=True, which did not hold with PyTorch below 2.6.0. Fixed in vLLM 0.8.0; the remedy is PyTorch 2.6.0 (see CVE-2025-32434). The GitHub Advisory Database rates it critical; the vLLM repository’s own advisory rates it high.

The exploitation condition for both is the same: the server has to load a malicious pickle-based checkpoint. Keep that in mind; it decides the whole investigation.

The system: Northwind’s Triage Assistant

Northwind Health (fictional) serves northwind/triage-assistant 2.3.0, a LoRA fine-tune of mistralai/Mistral-7B-Instruct-v0.3, with vLLM 0.6.3 on three nodes. The model reads from a retrieval index, northwind-helpcentre-index, and the application calls an MCP tool server, ticketing-mcp, to read and write tickets. A human agent approves every draft. Here is the dependency graph from its AIBOM:

[
  {
    "ref": "app:triage-assistant@2.3.0",
    "dependsOn": [
      "model:northwind/triage-assistant@2.3.0",
      "pkg:pypi/vllm@0.6.3",
      "service:northwind-helpcentre-index",
      "service:ticketing-mcp"
    ]
  },
  {
    "ref": "model:northwind/triage-assistant@2.3.0",
    "dependsOn": ["model:mistralai/Mistral-7B-Instruct-v0.3"]
  },
  {
    "ref": "pkg:pypi/vllm@0.6.3",
    "dependsOn": [
      "pkg:pypi/torch@2.4.0",
      "pkg:pypi/transformers@4.45.2",
      "pkg:pypi/safetensors@0.4.5"
    ]
  }
]

Excerpt: the dependencies with children from Northwind’s CycloneDX 1.7 file. Every value is as in the full file, which is in the Pro AIBOM Template Library.

The application depends on the model, vLLM, the index and the tool server; vLLM depends on torch 2.4.0. That graph is your scoping map.

The timeline

Play it as if GHSA-ggpf-24jw-3fcw has just reached your feed. Times are relative; the findings are Northwind’s, and fictional.

Fictional example. Northwind Health, Triage Assistant and its datasets, services, scores and URLs are invented for illustration; the base model and open-source libraries are real.

Three lessons from the timeline

• An AIBOM records intent; the deploy log records fact. Northwind’s earlier VEX said not_affected because of a configuration. The AIBOM was right about the release and wrong about one host. A requires_configuration verdict is a claim to test on every node, not a reason to close the ticket.
• Know when the vulnerable code runs. Weights are loaded at process start. That turns “were we exploited at any time?” into “what was in the model directory at each start?”, which logs can answer.
• Scope from the services list, confirm with the network. The AIBOM lists what the system is designed to reach and marks the trust boundary. Network policy and flow logs tell you what a compromised process could actually reach and what it tried.

One thing the timeline could not prove: that no malicious file was present and later removed between starts. Northwind had file audit events on the model directory to rule that out. Without them, say so in the report.

What to pull and preserve

Collect these before you restart or redeploy anything. A redeploy overwrites the model directory and the process history you need. Prompt logs hold confidential customer text; handle them under your evidence and privacy rules.

Scoping the blast radius

If the check had failed, for example a .bin file found on the failover node, scope in this order:

1. The host. Treat it as compromised: isolate it, capture disk and memory, and rotate every secret the process could read (model registry tokens, service credentials).
2. The data in the process. Prompts carried ticket text that the AIBOM classifies as confidential, which may mention health context. If personal data may have been exposed, your privacy team needs the facts now: GDPR Article 33 gives 72 hours from awareness to notify the supervisory authority, where notification is required.
3. The services. For each service in the AIBOM, check reachability and flow logs. ticketing-mcp is the one to worry about: it crosses a trust boundary and holds write permission on tickets.
4. The source of the file. Who wrote it, from where, and whether other models came from the same place. The model registry and pipeline logs answer this.
5. The family tree. If the bad file came from a third-party model, query every AIBOM for that model as an ancestor (CycloneDX pedigree.ancestors, SPDX descendantOf). Every fine-tune, adapter, merge and quantisation of it is in scope. The lineage guide shows the fields.

A quick inventory query over a folder of CycloneDX files:

jq -r '.metadata.component.name as $app
  | .components[]
  | select((.purl // "") | test("^pkg:pypi/(vllm|torch)@"))
  | [$app, .purl] | @tsv' *.cdx.json

Updating the AIBOM and VEX afterwards

Close the incident by making the AIBOM true again. Northwind adds a VEX entry for the bypass advisory:

{
  "bom-ref": "vuln:GHSA-ggpf-24jw-3fcw",
  "id": "GHSA-ggpf-24jw-3fcw",
  "source": {
    "name": "GitHub Advisory Database",
    "url": "https://github.com/advisories/GHSA-ggpf-24jw-3fcw"
  },
  "references": [
    {
      "id": "CVE-2025-24357",
      "source": { "name": "NVD", "url": "https://nvd.nist.gov/vuln/detail/CVE-2025-24357" }
    }
  ],
  "ratings": [{ "source": { "name": "GitHub Advisory Database" }, "severity": "critical" }],
  "description": "CVE-2025-24357: Malicious model remote code execution fix bypass with PyTorch < 2.6.0",
  "affects": [{ "ref": "pkg:pypi/vllm@0.6.3" }, { "ref": "pkg:pypi/torch@2.4.0" }],
  "analysis": {
    "state": "not_affected",
    "justification": "requires_configuration",
    "response": ["update"],
    "detail": "FICTIONAL EXAMPLE. Every serving node runs --load-format safetensors (checked against deploy logs, incident NW-IR-0142) and the model directory holds only the signed .safetensors file. Upgrade to vLLM >= 0.8.0 and torch >= 2.6.0 planned."
  }
}

Written for this post. The advisory IDs, title and rating are real; the analysis is Northwind’s fictional verdict. It validates against the CycloneDX 1.7 schema inside the full Northwind BOM.

• State and justification. not_affected with requires_configuration is honest only because the configuration is now enforced on every node. During the incident it was in_triage.
• Response. update records the decision to upgrade. When vLLM 0.8.0 or later and torch 2.6.0 or later ship, the next AIBOM version lists the new components and the entry no longer applies.
• The setting itself. Record the load format as a property on the vLLM component, so the next responder can see what the verdict depends on.
• A new BOM version. Bump version, keep the serialNumber, and compare the two with the AIBOM Diff (Pro).

The security and runtime guide explains each of these fields and the SPDX equivalents.

Before the next incident

• Log the model version, or the AIBOM serial number, with every request.
• Make deploy logs show the arguments each inference process started with.
• Verify the model signature before the server loads anything. The Model Signing and Provenance Lab (Pro) practises catching a tampered model, and sign a model and attach provenance shows the setup.
• Watch the model directory for writes.
• Get AIBOMs from your vendors too, so the same queries work on bought AI. The AIBOM questions to ask every AI vendor lists what to request.
• Rehearse it. The AI Incident Response Checklist covers the wider AI incident, and the GRC side of the record is in the AIBOM as compliance evidence.

All guides by role are on the AIBOM Library hub.

Frequently asked questions

How do I find which AI model version was running during an incident?

Join two records. The AIBOM says what each release contains: model version, weights hash, runtime versions, tools and data. The deploy log says which release ran on which host from when to when. If your application logs the model version or AIBOM serial number with each request, you can go straight from a suspicious output to the exact release.

How do I check if we are affected by a vulnerability in vLLM or PyTorch?

Query your AIBOMs for the package URLs, for example pkg:pypi/vllm and pkg:pypi/torch, and compare the versions with the advisory’s affected range. Then check the advisory’s exploitation conditions against the deployed configuration, not just the BOM. For the vLLM torch.load advisories, the condition is loading a malicious pickle-based checkpoint.

What is VEX and how is it used in incident response?

VEX (Vulnerability Exploitability eXchange) is a statement of whether a known vulnerability affects a product and why. In CycloneDX it is the analysis on a vulnerabilities entry: state (for example in_triage, exploitable, not_affected, resolved), justification and detail. During an incident you set it to in_triage, then record the verdict and the evidence behind it.

What evidence should I preserve in an AI incident?

The AIBOM version live at the time, deploy manifests and process arguments per host, a listing and hashes of the model directory, the signature verification result, container image digests, process start events, network flow logs from the inference hosts, prompt and output logs for the window, the retrieval index build that was live, and the tool server’s audit log.

Can a malicious AI model run code when it is loaded?

Yes, if it is in a pickle-based format such as a PyTorch .bin or .pt file and the loader unpickles it. In 2024 JFrog reported about 100 models on Hugging Face with malicious functionality. The safetensors format stores tensors only, so loading it does not run code.

How do I find every system built on a compromised base model?

Query the lineage fields across your AIBOMs: CycloneDX pedigree.ancestors, SPDX descendantOf relationships, or base_model on Hugging Face cards. Every fine-tune, adapter, merge and quantisation that names the compromised model as an ancestor is in scope.

Sources

• GHSA-rh4j-5rhw-hr54 / CVE-2025-24357: vLLM torch.load RCE, fixed in 0.7.0 (published 27 January 2025)
• GHSA-ggpf-24jw-3fcw: CVE-2025-24357 fix bypass with PyTorch < 2.6.0, fixed in vLLM 0.8.0 (published 22 April 2025)
• GitHub Advisory Database: GHSA-ggpf-24jw-3fcw
• GHSA-53q9-r3pm-6pq6 / CVE-2025-32434: torch.load with weights_only=True RCE, fixed in PyTorch 2.6.0
• JFrog: Data Scientists Targeted by Malicious Hugging Face ML Models with Silent Backdoor (2024)
• Hugging Face Hub docs: pickle scanning
• CycloneDX 1.7 JSON reference (component, pedigree, modelCard, service, vulnerability analysis)
• sigstore/model-transparency: OpenSSF model signing
• Regulation (EU) 2016/679 (GDPR), Article 33

Checked on 29 October 2026. PlayCISO is not affiliated with any standards body, vendor or project named here. This is general information, not legal advice.