How to Audit a Machine Learning Model File
Downloading a model and loading it feels routine, but a pickle-based checkpoint can run attacker code the moment you deserialize it — and the dangerous step is the one that feels most harmless. This guide shows how to audit a model file for malicious payloads, provenance, and licensing before you ever load it.
In plain words
Auditing a machine learning model file means statically inspecting it for code-execution payloads — without loading it — and checking its provenance and license before you trust it, and PlayCISO has a free browser-based model file auditor, ModelAudit, to run that inspection.
Pickle and PyTorch checkpoints can run arbitrary code on load, so never open an untrusted file to look — scan it instead. Prefer safetensors wherever it exists, verify the source and hash, and treat every model as a supply-chain dependency.
Step by step
- 1
Identify the file format and how it deserializes
Check what kind of file you have before anything else. Pickle-based formats — raw .pkl, and PyTorch .pt/.bin checkpoints that wrap pickle — execute arbitrary Python when loaded, so loading an untrusted one can run attacker code on your machine. Formats like safetensors and GGUF store only tensors and metadata and cannot execute code. The format alone tells you how much to worry.
- 2
Never load an untrusted file to inspect it
The dangerous step is deserialization itself, so do not run torch.load or pickle.load on a file you have not vetted — that is exactly when a malicious payload fires. Inspect the file statically with a scanner that parses the pickle opcodes or archive structure without executing them. Treat "I just opened it to look" as the attack surface, because it is.
- 3
Scan for malicious payloads
Use a model scanner to look inside the file for signs of code execution: pickle opcodes that import modules or call functions (os.system, eval, exec, subprocess), suspicious globals, or embedded scripts. These are the mechanisms by which a weaponized checkpoint runs code during loading. A clean scan is not a guarantee, but a flagged one is a hard stop — do not load it.
- 4
Verify provenance and integrity
Establish where the file came from and that it has not been tampered with. Prefer models from the original author's official repository, check the publisher and download source, and verify a published hash or signature against the file you have. A model re-uploaded by an unknown third party, or one whose hash does not match, is a supply-chain red flag regardless of how clean the scan looks.
- 5
Check the license and usage terms
Read the model's license before you build on it. Many popular models carry terms that restrict commercial use, redistribution, or specific applications, and these live in the model card rather than any file metadata. Using a model outside its license is a legal exposure that a security scan will never catch, so make the license a required field in your audit.
- 6
Prefer safetensors and convert when you can
Where a safetensors version of the model exists, use it — it stores tensors without the code-execution risk of pickle, so it removes the most dangerous class of model-file attack entirely. If only a pickle checkpoint is available from a trusted source, convert it to safetensors in an isolated environment and distribute the safe version internally. Standardizing on safetensors is the strongest single control you can adopt.
- 7
Isolate loading and treat models as supply chain
Even after a clean audit, load new or third-party models in a sandboxed, least-privilege environment first rather than directly in production with real credentials. Pin exact versions, record each model in your inventory or AIBOM, and re-audit on every update, since a trusted model can be replaced by a malicious version upstream. A model file is a dependency, so give it the same supply-chain scrutiny as any other.
Upload or point ModelAudit at a model file and it statically inspects the serialization for code-execution payloads and unsafe pickle opcodes — without ever loading the file — free in your browser.
Related
How model-file auditing fits into securing the AI supply chain.
Build an AI Bill of MaterialsRecord the models you audit, with their provenance and licenses, in an AIBOM.
AI security risks explainedThe model-poisoning and supply-chain risks this audit defends against.
All PlayCISO toolsScanners and builders for AI, model, and supply-chain security.
Frequently asked questions
Why are pickle and PyTorch model files dangerous?
Python's pickle format can embed instructions that execute arbitrary code during deserialization, and PyTorch .pt/.bin checkpoints are built on pickle. That means simply loading an untrusted model file with torch.load or pickle.load can run attacker-controlled code on your machine with your privileges. The risk is in the act of loading, not in using the model afterward.
How can I inspect a model file safely?
Use a static scanner that parses the file's structure — the pickle opcodes or archive contents — without actually deserializing it. That lets you see imports, globals, and suspicious calls without triggering any embedded payload. The one thing to avoid is loading the file in Python to "take a look," because that is exactly the dangerous step.
What are safetensors and why are they safer?
Safetensors is a model file format that stores only tensor data and metadata, with no mechanism to execute code on load. Because there is no deserialization of arbitrary objects, the code-execution risk that pickle-based formats carry simply does not exist. When a safetensors version of a model is available, using it removes the most severe model-file attack class outright.
What does provenance mean for a model file?
Provenance is knowing where the file genuinely came from and being able to verify it has not been altered — the original author, the official repository, and a matching published hash or signature. A model re-hosted by an unknown third party, or one whose hash does not match the source, should be treated as suspect. Good provenance is what lets you trust a file a scanner says is clean.
Does a clean scan mean the model is completely safe?
No. A clean scan means no known malicious serialization patterns were found, which is valuable but not a guarantee against novel or well-hidden payloads, and it says nothing about the model's behavior, bias, or quality. Combine scanning with verified provenance, license checks, and loading new models in an isolated environment. Defense in depth, not a single green check.
Should I check a model's license during a security audit?
Yes. Licenses often restrict commercial use, redistribution, or particular applications, and those terms live in the model card rather than in any file your scanner reads. Using a model outside its license is a legal and business risk that security tooling will never flag, so make the license a required part of the audit alongside the technical checks.