AI Provenance Coverage Scorecard
Provenance is how you answer, after the fact, where did this come from and can I trust it? — about your training data, your model's output, your agent's actions and your supply chain. It is as important to AI security as injection defense. Score yourself across five areas and see where the record is thin. But provenance is a dial, not a switch: the scorecard rewards depth where stakes are high and warns you off signing every thumbnail, which is cost and false assurance with no payoff.
Input provenance
0Do you know where your training, RAG and context data came from? · weight 22
Output provenance
0Is AI-generated content signed, watermarked or disclosed? · weight 20
Decision & audit trail
0Who / what / when for every agent action? · weight 24
Supply-chain provenance
0Can you attest where model weights, prompts and tools came from? · weight 18
Third-party verifiability
0Can someone other than you check the claims? · weight 16
0/100
Little to no coverage
Fix these first
- 1.[Decision & audit trail] Every consequential agent action logs who/what it acted as, when, and on whose behalf.
- 2.[Decision & audit trail] The prompt, tools and model version behind each action are recorded.
- 3.[Decision & audit trail] Logs are tamper-evident (append-only / signed) and retained long enough to investigate.
- 4.[Input provenance] Every training / fine-tuning dataset has a recorded source, licence and collection date.
- 5.[Input provenance] RAG and retrieved context carry their source document and timestamp through to the answer.
A high score is not the goal on every artifact. Deep, verifiable provenance pays off on news, evidence, legal and financial outputs. Signing every internal thumbnail or ephemeral draft is cost and false assurance. Aim for strong coverage where it matters and lightweight disclosure elsewhere.
What this scorecard is
- A weighted 0-100 scorecard across five provenance areas: input provenance (where training, RAG and context data came from), output provenance (signing, watermarking and disclosure of AI output), decision and audit trail (who/what/when for agent actions), supply-chain provenance (model weights, prompts and tools — attestation and SLSA), and third-party verifiability.
- Each area gets its own bar and a prioritised list of the gaps that cost you the most points, so you fix the weakest link first instead of polishing an already-strong area.
- Provenance is a dial, not a switch: the scorecard explicitly reminds you to match provenance effort to stakes — full chain-of-custody belongs on news, evidence and high-trust outputs, not on every internal thumbnail or ephemeral draft.
- Provenance is as important as injection defense for AI security — it is how you answer "where did this come from and can I trust it?" after the fact — but blanket C2PA-signing everything is wasted effort and false assurance.
- Fully client-side and free. Your answers never leave the browser; export the scorecard as text.
Why this matters
When an AI system is challenged — a regulator asks how a decision was made, a journalist asks whether an image is real, an incident responder asks which agent ran which action — provenance is the only thing that can answer. A system with strong injection defense but no provenance can stop a bad action and still be unable to tell you what happened, who was affected, or whether its own training data was poisoned. The two controls are complementary, and provenance is the one teams routinely under-build because it pays off only later, under pressure.
The failure in the other direction is real too. Teams that discover provenance sometimes try to apply full cryptographic chain-of-custody to every artifact the system touches, which buries the signal in cost and creates a false impression that everything is equally trustworthy. The right posture is graded: deep, verifiable provenance on the high-trust and high-stakes artifacts, lightweight disclosure and logging on the rest. This scorecard measures coverage and nudges you toward that calibrated target.
Frequently asked questions
What is the AI Provenance Coverage Scorecard?
A free, in-browser assessment that scores how well your AI system tracks provenance — the verifiable record of where things came from and what happened to them. It covers five areas: input provenance (the origin of training, RAG and context data), output provenance (signing, watermarking and disclosure of generated content), the decision and audit trail (an attributable who/what/when for agent actions), supply-chain provenance (attestation for model weights, prompts and tools), and verification (whether a third party can independently check your claims). You get a weighted 0-100 score, per-area bars and a prioritised list of gaps.
Why does provenance matter for AI security?
Provenance answers the questions you only get asked after something goes wrong: where did this training data come from, is this content really AI-generated, which agent took this action and on whose behalf, and can anyone but us verify it? Without it you cannot investigate an incident, prove compliance, or defend against data-poisoning and content-spoofing. It is as foundational as prompt-injection defense — one stops a bad action, the other lets you understand and prove what happened.
Should everything have full provenance?
No, and the scorecard is explicit about this. Provenance is a dial, not a switch. Cryptographically signing and keeping chain-of-custody for a piece of news media, a legal exhibit or a high-trust financial output is worth the cost. Doing the same for every internal thumbnail, every ephemeral draft and every low-stakes log line is wasted engineering and storage, and it creates a false sense of rigor. Match the depth of provenance to the stakes of the artifact — the scorecard weights its recommendations accordingly.
Is this a compliance assessment?
No. It is an educational self-assessment that encodes good provenance practice. Specific regimes — the EU AI Act transparency duties, content-labeling laws, sector record-keeping rules — may impose concrete provenance requirements that go beyond or differ from this rubric. Use the scorecard to find and prioritise your gaps, then confirm the specifics with your compliance and legal owners.
This tool is free and always will be. Would a Pro workspace — saved designs, team sharing and exports — be worth paying for?
Show me Pro →No account needed to use this tool — this just tells us whether Pro is worth building out.Educational — general guidance, not security or legal advice.