Model Card Scorer
Paste a model card and see which sections are present, weak or missing, scored out of 20 against 10 fixed criteria, with a one-line fix for each. It reads Hugging Face YAML keys such as license, base_model, datasets and model-index, and ignores template placeholders.
Who this is for: AppSec and AI-security engineers deciding whether a model’s documentation is good enough to approve it, and GRC teams checking a card before it becomes evidence. Also for teams about to publish their own card. The rubric is below; Pro runs it on your card and adds the downloads.
What this tool is
- Paste a model card (a Hugging Face README.md with its YAML header, or any markdown or text card) and get a score out of 20 against 10 fixed criteria, each 0, 1 or 2.
- Each criterion shows what was found, whether it is present, weak or missing, and a one-line fix.
- Template placeholders such as "[More Information Needed]" are ignored, so an untouched template scores as empty.
- It also reports the facts an AIBOM needs from the card but the rubric does not score: base model id, adaptation method, weights format and signing.
- Downloads: PDF, CSV and JPG of the score, and a markdown skeleton with only the missing sections stubbed, using Hugging Face section names.
- It is a heuristic check of what the card says, run in your browser. It cannot tell whether the card is true.
The rubric
PlayCISO Model Card Rubric v1.0. Score only what is on the card itself. The same rubric scores the public cards in Real model cards, reviewed and scored, so your result is comparable.
| Criterion | 2: specific | 1: partial or vague | 0: absent |
|---|---|---|---|
| 1. Identity and versionDoes the card say who built the model, what it is, and which exact version this is? | Developer, model name, a version or release date, and model type/modality are all stated. | Some of these are stated (e.g. name and developer, but no version or date). | Only a name, or nothing beyond the repository title. |
| 2. Licence and termsIs the licence identified unambiguously? | Licence identified unambiguously: a standard SPDX id (e.g. apache-2.0, mit) in the text or metadata, or a custom licence named AND linked to its full text. | A custom licence named without a link, or a vague mention ('open licence', 'see terms') with nothing identifiable. | No licence information. |
| 3. Intended useDoes the card say what the model is for and who should use it? | Specific intended uses AND intended users or deployment context. | A generic purpose statement only (e.g. 'for research and commercial use'). | No intended-use statement. |
| 4. Out-of-scope and prohibited usesDoes the card say what the model must not be used for? | Specific out-of-scope or prohibited uses listed (on the card or in a linked policy the card names). | A generic caution only (e.g. 'use responsibly'). | Nothing on out-of-scope use. |
| 5. Training dataDoes the card describe what the model was trained on? | Sources identified specifically (named datasets, or named source categories such as web crawl, code, books, synthetic data with how it was made) AND at least one of: size (tokens/examples), date range or knowledge cutoff. | Only a generic description (e.g. 'a mix of publicly available data'), even if size or cutoff is given; or size/cutoff with no description of sources. | No training-data information. |
| 6. Evaluation resultsAre there evaluation results a reader can check? | Named benchmarks or datasets WITH numeric results, and enough setup to interpret them (e.g. shots, metric, or comparison models). | Claims of performance without numbers, or numbers without naming what was measured. | No evaluation results. |
| 7. Limitations, bias and risksDoes the card name the model's limitations and risks specifically? | Specific limitations, bias or failure modes for this model (e.g. unsupported languages, weak domains, measured hallucination rates). | Generic caveats only (e.g. 'may produce inaccurate or offensive output'), or a single partial caveat. | No limitations section. |
| 8. Safety testing and mitigationsDoes the card say how the model was safety-tested and what mitigations exist? | Describes safety evaluation or red-teaming (what was tested) AND mitigations or safeguards. | Mentions safety or red-teaming without specifics, or only says it has no moderation. | No safety information. |
| 9. Technical specificationsCan an engineer tell what the model is technically? | Architecture AND parameter count AND at least one of: context length, training compute/hardware, or energy/carbon. | Some technical detail (e.g. parameter count only). | No technical detail. |
| 10. Machine-readable metadata and feedback channelCan a tool parse the key facts, and is there a way to report problems? | Machine-readable metadata (e.g. Hugging Face YAML with license plus base_model, datasets or pipeline_tag) AND a named feedback or issue-reporting channel. | One of the two. | Neither. |
Bands
- Strong (17–20): Enough to fill the model and evaluation sections of an AIBOM without chasing the vendor.
- Adequate (12–16): Usable, with named gaps to close by asking the provider.
- Thin (7–11): A reviewer will need a questionnaire or a contract clause to fill the gaps.
- Poor (0–6): Treat the model as undocumented.
Worked example: two Northwind cards
Northwind Health and its Triage Assistant are fictional, used across the AIBOM Library. The base model, Mistral-7B-Instruct-v0.3, is real; the metric values are made up.
The first draft of the card is five short sections of prose. It scores 5/20 (Poor). It says the model is “built on Mistral” without an id, “trained on a mix of internal support data”, and “performs well on our internal tests”. Each of those reads like information and gives a reviewer nothing to check. Weak or missing: identity and version, licence and terms, intended use, out-of-scope and prohibited uses, training data, evaluation results, limitations, bias and risks, safety testing and mitigations, technical specifications, machine-readable metadata and feedback channel.
The finished card scores 20/20 (Strong). The differences are concrete: base_model: mistralai/Mistral-7B-Instruct-v0.3 in the YAML, the training set named with its size (48,210 tickets) and date range, a results table that names the eval set (northwind/triage-eval-v4, 1,200 tickets), named failure modes (English only, a weekly-refreshed RAG index that can lag a policy change), the red-team test and its mitigations, and an address for reporting problems.
| Criterion | First draft | Finished card |
|---|---|---|
| Identity and version | 0/2 | 2/2 |
| Licence and terms | 0/2 | 2/2 |
| Intended use | 1/2 | 2/2 |
| Out-of-scope and prohibited uses | 1/2 | 2/2 |
| Training data | 1/2 | 2/2 |
| Evaluation results | 1/2 | 2/2 |
| Limitations, bias and risks | 1/2 | 2/2 |
| Safety testing and mitigations | 0/2 | 2/2 |
| Technical specifications | 0/2 | 2/2 |
| Machine-readable metadata and feedback channel | 0/2 | 2/2 |
How the check works, and what it misses
- YAML first. It reads the front matter between the
---lines:license,license_link,base_model,datasets,pipeline_tag,languageandmodel-index(a model-index entry with a dataset and a metric value counts as checkable evaluation). - Then headings and labels. Markdown headings, bold labels such as “**Intended use:**” and plain-text headings, matched to each criterion by name.
- Then keywords. Numbers next to a named benchmark or eval set, parameter counts, context length, red-teaming, mitigations, “should not be used for” statements.
- Placeholders are removed first. “[More Information Needed]”, the template’s HTML comments and its stock sentences do not count.
- What it cannot do. It cannot follow links (a card that says “see our technical report” gets no credit for the report), read images or tables rendered as images, or tell whether a number is real. A card can use the right words and still say little; read the sections it marks as present before you rely on them.
From card to AIBOM
A card is the human-readable source for several AIBOM fields: intended use, limitations, evaluation, training data and the base model. The scorer also reports, without scoring, the facts an AIBOM needs that cards often skip: the exact base model id, how it was adapted, the weights format and whether the weights are signed. For what goes in each of those AIBOM sections, read evaluation, intended use, limitations and risk and identity, provenance and lineage. To compare the card formats themselves, see model card templates compared. A card that links a dataset card is stronger; build one in the Dataset Card Builder. Everything else is on the AIBOM Library hub.
Sources
Metadata keys and template sections were checked against these on 29 October 2026.
Frequently asked questions
How do I check if a model card is complete?
Score it against a fixed list. This tool uses 10 criteria: identity and version, licence, intended use, out-of-scope use, training data, evaluation results, limitations and bias, safety testing, technical specifications, and machine-readable metadata with a feedback channel. Each scores 0 (absent), 1 (vague) or 2 (specific). A card that scores 2 on a criterion gives you enough to act on without asking the provider.
What should a model card include?
At minimum: who built it and which version it is, the licence, what it is for and who uses it, what it must not be used for, what it was trained on, numeric evaluation results on named data, specific limitations and risks, how it was safety-tested, the architecture and size, and a contact. The Hugging Face template and the original Model Cards paper (Mitchell et al., 2019) cover most of this; the rubric adds safety testing and machine-readable metadata.
What does "[More Information Needed]" mean in a Hugging Face model card?
It is the default text the Hugging Face model card template puts in every field nobody filled in. Many published cards still carry it. The scorer strips it, along with the template’s HTML comments and stock sentences, before scoring, so a heading with only a placeholder under it counts as missing.
Is a model card the same as an AIBOM?
No. A model card is mostly prose for people. An AIBOM is a machine-readable inventory, for example a CycloneDX ML-BOM, that also records hashes, dependencies, datasets as components, signatures and runtime services. A good card is the best source for the AIBOM’s intended use, limitations and evaluation fields, and its base_model and datasets metadata map straight into the BOM.
How accurate is the score?
It is a heuristic. It looks for headings, YAML keys and keywords, so it can miss a section written in an unusual way, and it can credit a section that uses the right words but says little. It cannot check that a claim is true. Use it to find which parts of a card to read closely, and read them.
More free security tools
The tools people use most on PlayCISO.
Catch injected instructions hiding in content your AI agent should only read.
Score your readiness against the first international standard for AI security threats.
Critical CVEs across 21 major vendors, month by month — and what the 2026 surge means.
Build an ordered ruleset, replay labelled packets and catch shadowed and any-to-any rules.
Self-assess all six Functions, from Govern to Recover, and see your weakest one.
Search 172 threat actors across Microsoft, CrowdStrike, Mandiant and community names.