Open-Weight Model Security Standard: A Checklist
TL;DR · the short version
On September 17, 2026, Baseten's Base Labs partnered with Hugging Face and Goodfire AI to build public methods for training, evaluating and monitoring open-weight models, prompted by abliteration (removing a model's refusals) and 6,000+ abliterated models already on it. No specification or code exists yet. A credible standard should split controls by role with auditor-checkable evidence, tier them by capability, and map to existing frameworks. It cannot promise 'abliteration-proof' weights: anyone holding them can strip refusals.
On September 17, 2026, Baseten's research arm Base Labs announced a partnership with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastructure for open-weight models. The stated goal is to design safety into the whole model lifecycle — training, evaluation, monitoring, deployment — instead of adding it after release. What has not been announced is the standard itself: no spec, no paper, no code. It is an open call for contributions. That gap is the useful part. Before anyone writes the spec, it is worth being precise about what a security standard for open models has to contain to be worth adopting — and the one promise it should refuse to make.
What was actually announced
- Who does what: Hugging Face distributes models, Goodfire builds interpretability systems that inspect and shape behavior inside neural networks, and Baseten runs the inference infrastructure models are served on. Between them they cover publishing, understanding and serving.
- The trigger: abliteration — surgically removing the refusal behavior from a model's weights. Hugging Face already lists more than 6,000 abliterated models. We covered one of them, Dolphin3-Cyber-8B, earlier this week.
- The argument: openness gives better visibility into model behavior, so security controls can be transparent rather than trusted on faith.
- Not yet published: a name, evaluation criteria, monitoring mechanisms, a timeline, or any technical artifact.
How we built this checklist
Our first draft was the list most people would write: signed weights, safetensors only, model cards, pre-release red-teaming, abliteration resistance, interpretability monitoring, hub labels for modified models, a disclosure process, license terms. Every item sounds right. Critiquing it against how standards actually get adopted and audited, it fails in six ways:
- It promises the impossible. "Abliteration-resistant weights" is not achievable today. Whoever holds the weights can modify them; published tamper-resistance work raises the cost of stripping safeguards, it doesn't prevent it. A standard that promises resistance produces certificates that a weekend of work invalidates — worse than no standard, because it creates false confidence.
- It doesn't say who does what. Publishers, hubs, inference hosts and deploying organizations control completely different things. An unscoped list can't be audited, because nobody is accountable for any given line.
- It confuses intent with evidence. "Interpretability-based monitoring" names a research direction, not a control. Every requirement needs an artifact an auditor can check.
- It trusts provenance too far. A signature proves where weights came from, not that they are safe. In the August 2026 keyv/cacheable npm compromise, every malicious release carried a valid build attestation. Provenance is necessary, not sufficient.
- It relies on self-declaration. Asking uploaders to label their models "abliterated" catches only the honest ones. Detection has to be measured — refusal-rate evaluation, weight-diff against the declared base.
- It ignores the deploying organization and reinvents existing frameworks. Most real harm happens where a model is deployed, and a new standard that doesn't map to the NIST AI RMF, ISO/IEC 42001, the EU AI Act, OWASP and OpenSSF won't get used by anyone who already answers to those. It also treated a license's acceptable-use clause as a security control — it is a policy statement, not an enforcement mechanism.
The revised checklist below fixes each of these: controls are grouped by role, each names its evidence, obligations scale with model capability, and abliteration is handled by measurement and enforcement points rather than promises.
Tiering: not every model carries the same burden
| Tier | Rough definition | Obligation level |
|---|---|---|
| T1 | Small or narrow models; no meaningful uplift on high-risk capability evaluations | Provenance, safe format, model card |
| T2 | General-purpose models with measurable dual-use capability (e.g. cyber, persuasion) | T1 + pre-release evals, tamper-effort disclosure, lineage |
| T3 | Models with significant uplift on severe-harm evaluations (e.g. CBRN, autonomous cyber-offense) | T2 + third-party evaluation, staged release, incident reporting |
Tier should be decided by evaluation results, not parameter count alone — and re-decided whenever a derivative changes capability.
Useful next, if this was relevant to you
The checklist, by role
1. Model publishers
| Control | Evidence an auditor can check | Tier |
|---|---|---|
| Signed release with published hashes for every weight file | Signature verifiable with public tooling (e.g. OpenSSF model signing / Sigstore) | T1+ |
| Safe serialization only — no pickle-based formats | Release contains only safetensors (or equivalent non-executable) files | T1+ |
| Machine-readable model card and AI bill of materials | Base model, training-data summary, license, intended use, known limitations in a CycloneDX ML-BOM or equivalent | T1+ |
| Pre-release misuse and refusal evaluations | Published eval suite, version and scores; reproducible by a third party | T2+ |
| Tamper-effort disclosure — how much it takes to strip safeguards | Measured refusal rate before and after standard abliteration / fine-tuning attacks, with compute and data required | T2+ |
| Independent evaluation before release | Report from an evaluator with no commercial stake in the release | T3 |
| Vulnerability and incident disclosure channel | Published contact, response SLA, public record of fixed issues | T1+ |
2. Model hubs
| Control | Evidence | Tier |
|---|---|---|
| Measured, not self-declared, modification detection | Automated refusal-rate check and weight-diff against the declared base model on upload; result shown on the model page | T2+ |
| Lineage graph for derivatives | Every fine-tune, merge and quantization links to its parent; broken lineage is flagged | T1+ |
| Format and malware scanning on upload | Pickle/executable content blocked or flagged; scan result public | T1+ |
| Signature verification surfaced to downloaders | "Verified publisher" state tied to a checkable signature, not an account badge | T1+ |
| Takedown and escalation process for high-tier derivatives | Published policy and transparency reporting | T3 |
3. Inference hosts
| Control | Evidence | Tier |
|---|---|---|
| Weight integrity check at load time | Hash/signature verified against the publisher release before serving | T1+ |
| Runtime misuse monitoring that doesn't depend on the model refusing | Input/output classifiers or interpretability-based probes running outside the model; detection rates published | T2+ |
| Customer visibility into what is actually served | Exact model, version and modification status disclosed per endpoint | T1+ |
| Abuse logging and retention appropriate to tier | Documented retention, access controls and law-enforcement process | T2+ |
4. Deploying organizations
| Control | Evidence | Tier |
|---|---|---|
| Inventory of every open-weight model in use — including on laptops | Current register with owner, source, version and hash | All |
| Allowlist of approved sources and formats | Policy plus technical enforcement (proxy, registry mirror, endpoint control) | All |
| Own evaluation before production use | Refusal and misuse test results for your use case, not just the publisher's | T2+ |
| Isolation proportional to capability and data access | Network, data and tool access scoped per model; agent tools least-privilege | All |
| Re-review on any modification | Fine-tunes, merges and quantizations treated as new models with their own review | All |
Cross-cutting requirements
- Re-evaluation triggers. A new version, a capability jump in a derivative, or a disclosed vulnerability must reset the evidence — not just the date on the certificate.
- Mappings, not reinvention. Each control should reference the NIST AI RMF, ISO/IEC 42001, the EU AI Act's general-purpose AI obligations, the OWASP LLM Top 10 and OpenSSF model signing, so organizations can reuse evidence they already produce.
- Versioned and open. Public spec, public eval suites, public changelog. A standard for open models that is itself closed would undercut its own premise.
- Honest limits. The standard should say plainly what it cannot guarantee — starting with the fact that open weights can always be modified by whoever holds them.
What to do before the standard exists
- Build the inventory. Most organizations can't yet say which open-weight models are running on company hardware.
- Allow only safetensors-format models from vetted sources, and verify hashes against the upstream release.
- Run your own refusal and misuse evaluations before anything reaches production.
- Treat every fine-tune, merge or quantized derivative as a new, unreviewed model.
- Score refusal-removed models with a structured method before they enter the environment — the Abliterated Model Risk Calculator and Hugging Face Model Risk Scanner cover the first pass.
Templates and tools for this standard
For AI governance evidence under ISO/IEC 42001 and AI security under ISO/IEC 27090: who owns what, the plan, and the documents behind your claims.
RACI and project plan templates (each with a preview of its sample on the page):
- ISO 42001 RACI Template (Pro): Ten roles × clauses 4–10 and Annex A areas A.2–A.10.
- ISO 42001 Implementation Project Plan (Pro): Scope to Stage 2: phases, tasks, owners and indicative durations.
- ISO 27090 RACI Template (Pro): Sixteen roles × 94 AI security activities, checked for one Accountable each.
- ISO 27090 Project Plan Template (Pro): Editable phases, tasks, owners, deliverables and indicative durations.
Related ISO tools:
- AI Standards Crosswalk (Pro): 42001, 27001, 27090, EU AI Act and NIST AI RMF side by side.
- ISO 42001 SoA Builder (Pro): Statement of Applicability with status and justification.
- ISO 42001 Evidence Binder (Pro): Audit evidence by clause and Annex A area.
- ISO 27090 Alignment Statement: Safe "aligned with 27090" wording for trust pages and RFPs.
- Can You Be Certified to ISO 27090?: No. What can certify or attest AI security instead.
Every ISO tool on PlayCISO, grouped by standard: the AI standards toolkit.
The takeaway
Hub, interpretability and inference in one partnership is the right set of seats at the table: those are exactly the points where safeguards can be measured and enforced after weights leave the publisher. Whether the result is useful depends on choices not yet made. A standard that assigns controls by role, asks for evidence instead of assurances, scales with capability, and admits that open weights can always be modified will be adopted. One that certifies "abliteration-resistant" models will be broken the week it ships.
The same debate turned sharply political when Anthropic argued that Zhipu's open-weight GLM-5.3 is too cyber-capable to ship without gatekeeping — a framing worth taking seriously on capability and challenging on policy. See Anthropic's GLM-5.3 warning: right about the capability, contestable about the cure →.
PlayCISO is not affiliated with Baseten, Base Labs, Hugging Face or Goodfire AI. This article is based on public reporting (TechCrunch and others) as of September 24, 2026; no specification had been published at the time of writing. The checklist is PlayCISO's own proposal, not a description of the partnership's plans.
Frequently asked questions
What did Baseten, Hugging Face and Goodfire announce?
On September 17, 2026, Baseten's research group Base Labs announced a partnership with Hugging Face and Goodfire AI to develop and publish methods for training, evaluating and monitoring open-weight models, with safety checks built into the full lifecycle from training through deployment. Hugging Face contributes the distribution platform, Goodfire its interpretability research, and Baseten its inference infrastructure. No technical specification, paper or code has been released yet — it was announced as an open call for community contributions.
Why is abliteration the problem this targets?
Abliteration identifies the internal direction in a model's weights responsible for refusing requests and removes it, producing a model that answers what the original would decline without a full retrain. Hugging Face lists more than 6,000 abliterated models. Because it only needs access to the weights, every open-weight release is exposed to it.
Can a standard make open-weight models resistant to abliteration?
Not in any way that should be promised today. Whoever holds the weights can modify them, and published tamper-resistance techniques have so far raised the cost of removing safeguards rather than prevented it. A credible standard should require publishers to measure and disclose how much effort it takes to strip safeguards from their release, and should put real enforcement at the points that can actually be controlled: hubs, inference hosts and the organizations deploying models.
What should an open-weight model security standard contain?
Controls grouped by who can actually carry them out — model publishers, model hubs, inference hosts, and deploying organizations — each with a verifiable piece of evidence rather than a statement of intent; obligations tiered by model capability so small models aren't held to frontier-model requirements; triggers for re-evaluation when a model is fine-tuned or modified; and mappings to existing frameworks such as the NIST AI RMF, ISO/IEC 42001, the EU AI Act's general-purpose AI obligations, OWASP's LLM Top 10 and OpenSSF model signing.
What should a security team do before the standard exists?
Inventory which open-weight models are in use, including ones on individual laptops; allow only models from vetted sources in safe serialization formats; verify hashes or signatures against the upstream release; run your own refusal and misuse evaluations before production use; and treat any fine-tuned or modified derivative as a new model that needs its own review. PlayCISO's Abliterated Model Risk Calculator and Hugging Face Model Risk Scanner (Pro) cover the first pass.