Baseten, Hugging Face and Goodfire Want a Security Standard for Open-Weight Models. Here's What One Should Actually Contain
On September 17, 2026, Baseten's research arm Base Labs announced a partnership with Hugging Face and Goodfire AI to develop public methods for training, evaluating and monitoring open-weight models, building safety checks into the whole model lifecycle rather than bolting them on after release. The trigger is abliteration — surgically removing a model's refusal behavior — with more than 6,000 abliterated models already listed on Hugging Face. The three partners cover the chain: Hugging Face distributes models, Goodfire builds interpretability tools that inspect behavior inside a network, and Baseten serves models in production. No specification, paper or code has been published yet; it is an open call for contributions. That makes this the right moment to say what a credible standard should contain. Our answer: controls split by role (publisher, hub, inference host, deploying organization), each with evidence an auditor can check, tiered by model capability, mapped to frameworks that already exist — and an explicit refusal to promise "abliteration-proof" weights, because anyone holding the weights can remove refusals. A good standard measures and discloses how easily safeguards come off; it does not pretend they can't.
On September 17, 2026, Baseten's research arm Base Labs announced a partnership with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastructure for open-weight models. The stated goal is to design safety into the whole model lifecycle — training, evaluation, monitoring, deployment — instead of adding it after release. What has not been announced is the standard itself: no spec, no paper, no code. It is an open call for contributions. That gap is the useful part. Before anyone writes the spec, it is worth being precise about what a security standard for open models has to contain to be worth adopting — and the one promise it should refuse to make.
What was actually announced
- Who does what: Hugging Face distributes models, Goodfire builds interpretability systems that inspect and shape behavior inside neural networks, and Baseten runs the inference infrastructure models are served on. Between them they cover publishing, understanding and serving.
- The trigger: abliteration — surgically removing the refusal behavior from a model's weights. Hugging Face already lists more than 6,000 abliterated models. We covered one of them, Dolphin3-Cyber-8B, earlier this week.
- The argument: openness gives better visibility into model behavior, so security controls can be transparent rather than trusted on faith.
- Not yet published: a name, evaluation criteria, monitoring mechanisms, a timeline, or any technical artifact.
How we built this checklist
Our first draft was the list most people would write: signed weights, safetensors only, model cards, pre-release red-teaming, abliteration resistance, interpretability monitoring, hub labels for modified models, a disclosure process, license terms. Every item sounds right. Critiquing it against how standards actually get adopted and audited, it fails in six ways:
- It promises the impossible. "Abliteration-resistant weights" is not achievable today. Whoever holds the weights can modify them; published tamper-resistance work raises the cost of stripping safeguards, it doesn't prevent it. A standard that promises resistance produces certificates that a weekend of work invalidates — worse than no standard, because it creates false confidence.
- It doesn't say who does what. Publishers, hubs, inference hosts and deploying organizations control completely different things. An unscoped list can't be audited, because nobody is accountable for any given line.
- It confuses intent with evidence. "Interpretability-based monitoring" names a research direction, not a control. Every requirement needs an artifact an auditor can check.
- It trusts provenance too far. A signature proves where weights came from, not that they are safe. In the August 2026 keyv/cacheable npm compromise, every malicious release carried a valid build attestation. Provenance is necessary, not sufficient.
- It relies on self-declaration. Asking uploaders to label their models "abliterated" catches only the honest ones. Detection has to be measured — refusal-rate evaluation, weight-diff against the declared base.
- It ignores the deploying organization and reinvents existing frameworks. Most real harm happens where a model is deployed, and a new standard that doesn't map to the NIST AI RMF, ISO/IEC 42001, the EU AI Act, OWASP and OpenSSF won't get used by anyone who already answers to those. It also treated a license's acceptable-use clause as a security control — it is a policy statement, not an enforcement mechanism.
The revised checklist below fixes each of these: controls are grouped by role, each names its evidence, obligations scale with model capability, and abliteration is handled by measurement and enforcement points rather than promises.
Tiering: not every model carries the same burden
| Tier | Rough definition | Obligation level |
|---|---|---|
| T1 | Small or narrow models; no meaningful uplift on high-risk capability evaluations | Provenance, safe format, model card |
| T2 | General-purpose models with measurable dual-use capability (e.g. cyber, persuasion) | T1 + pre-release evals, tamper-effort disclosure, lineage |
| T3 | Models with significant uplift on severe-harm evaluations (e.g. CBRN, autonomous cyber-offense) | T2 + third-party evaluation, staged release, incident reporting |
Tier should be decided by evaluation results, not parameter count alone — and re-decided whenever a derivative changes capability.
The checklist, by role
1. Model publishers
| Control | Evidence an auditor can check | Tier |
|---|---|---|
| Signed release with published hashes for every weight file | Signature verifiable with public tooling (e.g. OpenSSF model signing / Sigstore) | T1+ |
| Safe serialization only — no pickle-based formats | Release contains only safetensors (or equivalent non-executable) files | T1+ |
| Machine-readable model card and AI bill of materials | Base model, training-data summary, license, intended use, known limitations in a CycloneDX ML-BOM or equivalent | T1+ |
| Pre-release misuse and refusal evaluations | Published eval suite, version and scores; reproducible by a third party | T2+ |
| Tamper-effort disclosure — how much it takes to strip safeguards | Measured refusal rate before and after standard abliteration / fine-tuning attacks, with compute and data required | T2+ |
| Independent evaluation before release | Report from an evaluator with no commercial stake in the release | T3 |
| Vulnerability and incident disclosure channel | Published contact, response SLA, public record of fixed issues | T1+ |
2. Model hubs
| Control | Evidence | Tier |
|---|---|---|
| Measured, not self-declared, modification detection | Automated refusal-rate check and weight-diff against the declared base model on upload; result shown on the model page | T2+ |
| Lineage graph for derivatives | Every fine-tune, merge and quantization links to its parent; broken lineage is flagged | T1+ |
| Format and malware scanning on upload | Pickle/executable content blocked or flagged; scan result public | T1+ |
| Signature verification surfaced to downloaders | "Verified publisher" state tied to a checkable signature, not an account badge | T1+ |
| Takedown and escalation process for high-tier derivatives | Published policy and transparency reporting | T3 |
3. Inference hosts
| Control | Evidence | Tier |
|---|---|---|
| Weight integrity check at load time | Hash/signature verified against the publisher release before serving | T1+ |
| Runtime misuse monitoring that doesn't depend on the model refusing | Input/output classifiers or interpretability-based probes running outside the model; detection rates published | T2+ |
| Customer visibility into what is actually served | Exact model, version and modification status disclosed per endpoint | T1+ |
| Abuse logging and retention appropriate to tier | Documented retention, access controls and law-enforcement process | T2+ |
4. Deploying organizations
| Control | Evidence | Tier |
|---|---|---|
| Inventory of every open-weight model in use — including on laptops | Current register with owner, source, version and hash | All |
| Allowlist of approved sources and formats | Policy plus technical enforcement (proxy, registry mirror, endpoint control) | All |
| Own evaluation before production use | Refusal and misuse test results for your use case, not just the publisher's | T2+ |
| Isolation proportional to capability and data access | Network, data and tool access scoped per model; agent tools least-privilege | All |
| Re-review on any modification | Fine-tunes, merges and quantizations treated as new models with their own review | All |
Cross-cutting requirements
- Re-evaluation triggers. A new version, a capability jump in a derivative, or a disclosed vulnerability must reset the evidence — not just the date on the certificate.
- Mappings, not reinvention. Each control should reference the NIST AI RMF, ISO/IEC 42001, the EU AI Act's general-purpose AI obligations, the OWASP LLM Top 10 and OpenSSF model signing, so organizations can reuse evidence they already produce.
- Versioned and open. Public spec, public eval suites, public changelog. A standard for open models that is itself closed would undercut its own premise.
- Honest limits. The standard should say plainly what it cannot guarantee — starting with the fact that open weights can always be modified by whoever holds them.
What to do before the standard exists
- Build the inventory. Most organizations can't yet say which open-weight models are running on company hardware.
- Allow only safetensors-format models from vetted sources, and verify hashes against the upstream release.
- Run your own refusal and misuse evaluations before anything reaches production.
- Treat every fine-tune, merge or quantized derivative as a new, unreviewed model.
- Score refusal-removed models with a structured method before they enter the environment — the free Abliterated Model Risk Calculator and Hugging Face Model Risk Scanner cover the first pass.
The takeaway
Hub, interpretability and inference in one partnership is the right set of seats at the table: those are exactly the points where safeguards can be measured and enforced after weights leave the publisher. Whether the result is useful depends on choices not yet made. A standard that assigns controls by role, asks for evidence instead of assurances, scales with capability, and admits that open weights can always be modified will be adopted. One that certifies "abliteration-resistant" models will be broken the week it ships.
PlayCISO is not affiliated with Baseten, Base Labs, Hugging Face or Goodfire AI. This article is based on public reporting (TechCrunch and others) as of September 24, 2026; no specification had been published at the time of writing. The checklist is PlayCISO's own proposal, not a description of the partnership's plans.
Ready to practise the decisions these articles describe?
Run a free War Room →