# AIBOM in Incident Response: Which Model Ran?

> A worked AI incident on a vLLM model-loading advisory: the timeline, what to preserve, how to scope the blast radius and the VEX entry to write after.

Source: https://playciso.com/blog/aibom-incident-response-which-model-was-running · Published: 2026-10-29 · Publisher: PlayCISO (https://playciso.com)

---

When an AI component is compromised or an advisory lands, the AIBOM answers what each release contains, and the deploy log answers which release ran where and when. Together they tell you which model was running, what it could reach and what data it saw. Below is a worked incident on a real class of issue, malicious model files that run code on load, played out on a fictional company. **Who this is for:** SOC analysts and incident responders who get the call, and the AppSec engineers who own the model pipeline and will be asked for the evidence. ## The issue: model files that run code when loaded A model saved in a pickle-based format (a PyTorch `.bin` or `.pt` file) can execute code when it is unpickled. This is not theoretical. In 2024 JFrog reported about 100 PyTorch and TensorFlow Keras models on Hugging Face with malicious functionality; the example it analysed ran its payload through pickle when the file was loaded. The safetensors format stores tensors only and does not run code on load. Inference servers have had advisories on exactly this path: - CVE-2025-24357 (GHSA-rh4j-5rhw-hr54, published 27 January 2025): vLLM called torch.load with weights_only left at False, so a malicious checkpoint could run code. Fixed in vLLM 0.7.0. The advisory notes that safetensors models are not vulnerable. - GHSA-ggpf-24jw-3fcw (published 22 April 2025): the 0.7.0 fix relied on weights_only=True, which did not hold with PyTorch below 2.6.0. Fixed in vLLM 0.8.0; the remedy is PyTorch 2.6.0 (see CVE-2025-32434). The GitHub Advisory Database rates it critical; the vLLM repository’s own advisory rates it high. The exploitation condition for both is the same: the server has to load a malicious pickle-based checkpoint. Keep that in mind; it decides the whole investigation. ## The system: Northwind’s Triage Assistant Northwind Health (fictional) serves `northwind/triage-assistant` 2.3.0, a LoRA fine-tune of `mistralai/Mistral-7B-Instruct-v0.3`, with vLLM 0.6.3 on three nodes. The model reads from a retrieval index, `northwind-helpcentre-index`, and the application calls an MCP tool server, `ticketing-mcp`, to read and write tickets. A human agent approves every draft. Here is the dependency graph from its AIBOM: `[ { "ref": "app:triage-assistant@2.3.0", "dependsOn": [ "model:northwind/triage-assistant@2.3.0", "pkg:pypi/vllm@0.6.3", "service:northwind-helpcentre-index", "service:ticketing-mcp" ] }, { "ref": "model:northwind/triage-assistant@2.3.0", "dependsOn": ["model:mistralai/Mistral-7B-Instruct-v0.3"] }, { "ref": "pkg:pypi/vllm@0.6.3", "dependsOn": [ "pkg:pypi/torch@2.4.0", "pkg:pypi/transformers@4.45.2", "pkg:pypi/safetensors@0.4.5" ] } ]` _Excerpt: the dependencies with children from Northwind’s CycloneDX 1.7 file. Every value is as in the full file, which is in the Pro [AIBOM Template Library](/tools/aibom-template-library)._ The application depends on the model, vLLM, the index and the tool server; vLLM depends on torch 2.4.0. That graph is your scoping map. ## The timeline Play it as if GHSA-ggpf-24jw-3fcw has just reached your feed. Times are relative; the findings are Northwind’s, and fictional. TimeQuestionWhere the answer isNorthwind finding (fictional) T+0:00Which of our systems run vLLM below 0.8.0 with PyTorch below 2.6.0?AIBOM inventory: query `purl` for `pkg:pypi/vllm` and `pkg:pypi/torch`One system: Triage Assistant 2.3.0 (`pkg:pypi/vllm@0.6.3`, `pkg:pypi/torch@2.4.0`). T+0:15What does our AIBOM already say about this class of issue?The `vulnerabilities` entries and their `analysis`CVE-2025-24357: `not_affected`, `requires_configuration` (vLLM pinned to `--load-format safetensors`). No entry yet for the fix bypass. Set it to `in_triage`. T+0:30Is that configuration true on every node, right now and for the whole window?Deploy logs, orchestrator manifests, running process argumentsTwo of three nodes pinned. The failover node, added during a capacity change, runs the default load format, which falls back to .bin when no safetensors file is present. T+1:00What did the failover node load, and when?Process start events; model directory listing; SHA-256 against the AIBOM; signature verificationWeights load only at process start, so the window is the start events. The directory holds only the model file; its hash matches the AIBOM and the Sigstore signature verifies against the expected CI identity. No .bin file. T+1:30If code had run in that process, what could it reach?AIBOM `services` and `dependencies`, then network policy and flow logs`northwind-helpcentre-index`: reachable (public data). `ticketing-mcp`: crosses a trust boundary and holds `tickets:read, tickets:write`; network policy blocks it from the inference subnet and flow logs show no attempts. Prompts carry confidential ticket text. T+2:00ContainChange recordFailover node pinned to safetensors and restarted. A pipeline check now fails any deploy without the pin. T+4:00DecideIncident recordExposure through configuration drift, no evidence of exploitation. Facts handed to the privacy team for its own assessment. T+1 dayUpdate the recordAIBOM and VEXVEX entry added (below), new AIBOM version issued, upgrade scheduled. _Fictional example. Northwind Health, Triage Assistant and its datasets, services, scores and URLs are invented for illustration; the base model and open-source libraries are real._ ## Three lessons from the timeline - An AIBOM records intent; the deploy log records fact. Northwind’s earlier VEX said not_affected because of a configuration. The AIBOM was right about the release and wrong about one host. A requires_configuration verdict is a claim to test on every node, not a reason to close the ticket. - Know when the vulnerable code runs. Weights are loaded at process start. That turns “were we exploited at any time?” into “what was in the model directory at each start?”, which logs can answer. - Scope from the services list, confirm with the network. The AIBOM lists what the system is designed to reach and marks the trust boundary. Network policy and flow logs tell you what a compromised process could actually reach and what it tried. One thing the timeline could not prove: that no malicious file was present and later removed between starts. Northwind had file audit events on the model directory to rule that out. Without them, say so in the report. ## What to pull and preserve EvidenceWhat exactlyWhat it answers AIBOM live at the time`serialNumber` and `version` of the BOM for the running releaseWhat the system was supposed to be Deploy recordManifests, process arguments and image digests per host, with timestampsWhat actually ran, where and when Model directoryFile listing, SHA-256 of every file, signature verification outputWhether the bytes match the signed release Process startsStart and restart events for the inference serverWhen weights were loaded (the only time a malicious checkpoint can run) NetworkFlow logs from the inference hosts to each service in the AIBOM and to the internetWhat a compromised process tried to reach Prompts and outputsRequest logs for the window, with the model version per requestWhat data the model saw and produced (handle as confidential) RetrievalThe `northwind-helpcentre-index` build that was live (rebuilt weekly)What content went into prompts Tool callsThe `ticketing-mcp` audit logWhat the model did, not just what it said Collect these before you restart or redeploy anything. A redeploy overwrites the model directory and the process history you need. Prompt logs hold confidential customer text; handle them under your evidence and privacy rules. ## Scoping the blast radius If the check had failed, for example a `.bin` file found on the failover node, scope in this order: - The host. Treat it as compromised: isolate it, capture disk and memory, and rotate every secret the process could read (model registry tokens, service credentials). - The data in the process. Prompts carried ticket text that the AIBOM classifies as confidential, which may mention health context. If personal data may have been exposed, your privacy team needs the facts now: GDPR Article 33 gives 72 hours from awareness to notify the supervisory authority, where notification is required. - The services. For each service in the AIBOM, check reachability and flow logs. ticketing-mcp is the one to worry about: it crosses a trust boundary and holds write permission on tickets. - The source of the file. Who wrote it, from where, and whether other models came from the same place. The model registry and pipeline logs answer this. - The family tree. If the bad file came from a third-party model, query every AIBOM for that model as an ancestor (CycloneDX pedigree.ancestors, SPDX descendantOf). Every fine-tune, adapter, merge and quantisation of it is in scope. The lineage guide shows the fields. A quick inventory query over a folder of CycloneDX files: `jq -r '.metadata.component.name as $app | .components[] | select((.purl // "") | test("^pkg:pypi/(vllm|torch)@")) | [$app, .purl] | @tsv' *.cdx.json` ## Updating the AIBOM and VEX afterwards Close the incident by making the AIBOM true again. Northwind adds a VEX entry for the bypass advisory: `{ "bom-ref": "vuln:GHSA-ggpf-24jw-3fcw", "id": "GHSA-ggpf-24jw-3fcw", "source": { "name": "GitHub Advisory Database", "url": "https://github.com/advisories/GHSA-ggpf-24jw-3fcw" }, "references": [ { "id": "CVE-2025-24357", "source": { "name": "NVD", "url": "https://nvd.nist.gov/vuln/detail/CVE-2025-24357" } } ], "ratings": [{ "source": { "name": "GitHub Advisory Database" }, "severity": "critical" }], "description": "CVE-2025-24357: Malicious model remote code execution fix bypass with PyTorch = 0.8.0 and torch >= 2.6.0 planned." } }` _Written for this post. The advisory IDs, title and rating are real; the analysis is Northwind’s fictional verdict. It validates against the CycloneDX 1.7 schema inside the full Northwind BOM._ - State and justification. not_affected with requires_configuration is honest only because the configuration is now enforced on every node. During the incident it was in_triage. - Response. update records the decision to upgrade. When vLLM 0.8.0 or later and torch 2.6.0 or later ship, the next AIBOM version lists the new components and the entry no longer applies. - The setting itself. Record the load format as a property on the vLLM component, so the next responder can see what the verdict depends on. - A new BOM version. Bump version, keep the serialNumber, and compare the two with the AIBOM Diff (Pro). The [security and runtime guide](/blog/aibom-security-runtime-section) explains each of these fields and the SPDX equivalents. ## Before the next incident - Log the model version, or the AIBOM serial number, with every request. - Make deploy logs show the arguments each inference process started with. - Verify the model signature before the server loads anything. The Model Signing and Provenance Lab (Pro) practises catching a tampered model, and sign a model and attach provenance shows the setup. - Watch the model directory for writes. - Get AIBOMs from your vendors too, so the same queries work on bought AI. The AIBOM questions to ask every AI vendor lists what to request. - Rehearse it. The AI Incident Response Checklist covers the wider AI incident, and the GRC side of the record is in the AIBOM as compliance evidence. All guides by role are on the [AIBOM Library hub](/aibom). ## Frequently asked questions ### How do I find which AI model version was running during an incident? Join two records. The AIBOM says what each release contains: model version, weights hash, runtime versions, tools and data. The deploy log says which release ran on which host from when to when. If your application logs the model version or AIBOM serial number with each request, you can go straight from a suspicious output to the exact release. ### How do I check if we are affected by a vulnerability in vLLM or PyTorch? Query your AIBOMs for the package URLs, for example pkg:pypi/vllm and pkg:pypi/torch, and compare the versions with the advisory’s affected range. Then check the advisory’s exploitation conditions against the deployed configuration, not just the BOM. For the vLLM torch.load advisories, the condition is loading a malicious pickle-based checkpoint. ### What is VEX and how is it used in incident response? VEX (Vulnerability Exploitability eXchange) is a statement of whether a known vulnerability affects a product and why. In CycloneDX it is the analysis on a vulnerabilities entry: state (for example in_triage, exploitable, not_affected, resolved), justification and detail. During an incident you set it to in_triage, then record the verdict and the evidence behind it. ### What evidence should I preserve in an AI incident? The AIBOM version live at the time, deploy manifests and process arguments per host, a listing and hashes of the model directory, the signature verification result, container image digests, process start events, network flow logs from the inference hosts, prompt and output logs for the window, the retrieval index build that was live, and the tool server’s audit log. ### Can a malicious AI model run code when it is loaded? Yes, if it is in a pickle-based format such as a PyTorch .bin or .pt file and the loader unpickles it. In 2024 JFrog reported about 100 models on Hugging Face with malicious functionality. The safetensors format stores tensors only, so loading it does not run code. ### How do I find every system built on a compromised base model? Query the lineage fields across your AIBOMs: CycloneDX pedigree.ancestors, SPDX descendantOf relationships, or base_model on Hugging Face cards. Every fine-tune, adapter, merge and quantisation that names the compromised model as an ancestor is in scope. ## Sources - GHSA-rh4j-5rhw-hr54 / CVE-2025-24357: vLLM torch.load RCE, fixed in 0.7.0 (published 27 January 2025) - GHSA-ggpf-24jw-3fcw: CVE-2025-24357 fix bypass with PyTorch - GitHub Advisory Database: GHSA-ggpf-24jw-3fcw - GHSA-53q9-r3pm-6pq6 / CVE-2025-32434: torch.load with weights_only=True RCE, fixed in PyTorch 2.6.0 - JFrog: Data Scientists Targeted by Malicious Hugging Face ML Models with Silent Backdoor (2024) - Hugging Face Hub docs: pickle scanning - CycloneDX 1.7 JSON reference (component, pedigree, modelCard, service, vulnerability analysis) - sigstore/model-transparency: OpenSSF model signing - Regulation (EU) 2016/679 (GDPR), Article 33 _Checked on 29 October 2026. PlayCISO is not affiliated with any standards body, vendor or project named here. This is general information, not legal advice._