Training Data Quality & Provenance Checklist
Prove where your training and fine-tuning data came from, that you are allowed to use it, that it is good enough and that nobody poisoned it — with a dataset register that flags anything you should not train on yet.
Template & sample only · See the sample ↓ — a structured starting point with sample content, not legal, audit, certification or investment advice.
For a model you train or fine-tune, prove where every dataset came from, that you are allowed to use it, that it is good enough, and that nobody tampered with it. The checklist follows the data-quality themes of ISO/IEC 5259 and the data life-cycle stages of ISO/IEC 8183; the dataset register records each dataset's source, basis, personal data and checks. The summary flags datasets you should not train on yet.
- Special category and children's data is excluded, or its use is specifically justified and approved
Your context
Dataset register
One row per dataset used for training, fine-tuning or evaluation. Write "Unknown" in the basis column rather than leaving a guess.
Lawful basis & licensing
Every dataset has a documented right to be used for model training, and licence terms permit this use (including commercial use and derived models)5259 / 8183material
Use of customer or user data for training is permitted by customer contracts and privacy noticesmaterial
Web-collected data respects site terms and machine-readable opt-outs from text and data mining, and the collection method is documentedEU AI Act
Purchased or third-party datasets come with supplier warranties on provenance and rights42001 A.10
The base model's licence and acceptable-use terms permit your fine-tuning and intended use, and its own training-data disclosures were reviewed
Provenance records
A provenance record exists per dataset: source, collection method, collection dates, who collected it and every transformation applied8183material
Each dataset has a datasheet or data card describing purpose, composition, known gaps and intended uses
Synthetic and model-generated data is labelled as such, with the generating model, prompts and filtering recorded
Consent & personal data
A lawful basis is identified for each personal-data dataset, and a DPIA covers the training useGDPRmaterial
Where consent is the basis, consent records are linkable to the data and withdrawal is handled, including what it means for models already trained
Direct identifiers are removed or pseudonymised, and free-text fields are scanned for personal data before trainingmaterial
Special category and children's data is excluded, or its use is specifically justified and approvedmaterial
The trained model is tested for memorisation and extraction of sensitive training records
Labelling quality
Written labelling guidelines with worked examples and edge cases are issued to every labeller
Agreement between labellers is measured and disagreements are adjudicated5259
Labelling vendors are vetted for security, data access and working conditions, and contracted not to retain data42001 A.10
Bias & representativeness
Representativeness is assessed against the population the model will serve, and gaps are documented with their likely effect5259material
Features that may act as proxies for protected characteristics (postcode, name, language) are identified and their use justified
Data-level measures (class balance, subgroup counts, missing-value rates by group) are recorded per dataset version
Poisoning defences
Data is ingested only from authenticated, allow-listed sources, with integrity hashes checked at each stage27090material
Write access to training data stores is restricted and logged, and promotion of a new dataset version needs a second approvermaterial
Outlier, duplicate and suspicious-pattern screening runs before training, including checks for planted trigger phrases
User feedback and other loops that feed back into training are protected against deliberate manipulation
Versioning & lineage
Dataset versions are immutable and content-hashed, and every model records exactly which versions trained and evaluated it8183material
Preparation pipelines are in version control and can reproduce a training set from raw sources
Lineage from raw data through processed data to training set and model is available to auditors on request
Retention & deletion
Each dataset has a retention period consistent with its lawful basis, and expiry triggers review or deletion8183material
Erasure requests reach training stores, with a documented approach for models already trained on the data (retrain, unlearn or justify)
Disposal covers derived copies — caches, feature stores, embeddings, backups, vendor copies — with evidence
Loading…
Sample report · illustrative data
Sample result: training data provenance register
Example register for an invented claims-summarisation model fine-tuned from a foundation model: five datasets, each with its source, lawful basis, personal-data class, bias check and retention date.
Dataset register (excerpt)
| Dataset | Licence / lawful basis | Personal data | Bias check | Retain until |
|---|---|---|---|---|
| Historic claims | Legitimate interests; privacy notice covers model improvement | Pseudonymised | Partial | End of retention period set |
| Handler summary gold set | Internal work product | Pseudonymised | Done | End of retention period set |
| Medical correspondence sample | TBC, DPO review pending | Special category data | Not done | Not set |
| Public weather events | Open government licence (commercial use permitted) | None | Done | While model in service |
| Synthetic edge-case claims | Internal; generator model and prompts recorded | None | Partial | Date set |
Summary figures
- Datasets registered
- 5
- Unconfirmed licence or lawful basis
- 1 (do not train on it until resolved)
- Containing personal data, including pseudonymised
- 3
- Containing special category data
- 1 (needs specific justification and a DPIA)
- Bias check not complete
- 3
- No retention or deletion date
- 1
Checklist items rated (excerpt)
| Section | Item | Status |
|---|---|---|
| Lawful basis and licensing | Customer data use is permitted by contracts and privacy notices | In place |
| Consent and personal data | Special category data is excluded or specifically justified and approved | Not in place |
| Poisoning defences | Data is ingested only from allow-listed sources, with hashes checked at each stage | Partially in place |
| Versioning and lineage | Dataset versions are immutable and content-hashed | In place |
Made-up example data. Your own run uses your inputs. Downloads of the full report need Pro.
TL;DRWhat this template gives you
4 points
- Twenty-nine checks across licensing, provenance, personal data, labelling, bias, poisoning, versioning and deletion — for models you train or fine-tune.
- ISO/IEC 5259 covers data quality for analytics and machine learning; ISO/IEC 8183 describes the stages data passes through in an AI system. The checks follow both, in plain language.
- A dataset register records source, licence or lawful basis, personal data, version hash, labelling method, bias check and retention per dataset.
- The summary names datasets with an unconfirmed basis, special category data, missing bias checks or no deletion date.
Frequently asked questions
What is training data provenance?
It is the documented history of a dataset: where it came from, how and when it was collected, who handled it, what was done to it, and on what basis you may use it. Provenance lets you answer a regulator, a customer or a court asking "what was this model trained on, and were you allowed to?" — and lets you find and remove bad data after the fact.
What are ISO/IEC 5259 and ISO/IEC 8183?
ISO/IEC 5259 is a multi-part series on data quality for analytics and machine learning — how to define, measure and manage the quality of data used to build models. ISO/IEC 8183 is a framework describing the life cycle of data in AI systems, from the idea for a dataset through collection, preparation, use and eventual decommissioning. This template draws on both in its own words; it does not reproduce them.
Does this apply if I only use retrieval (RAG) and do not train?
Partly. Licensing, personal data, poisoning defences and retention still apply to the documents you retrieve from; labelling and training-specific bias checks may not. Mark the items that do not apply as N/A.
Is this a finished document I can rely on?
No. Every tool in this suite is a template with sample content. It gives you a structured, editable starting point and generates a document from your own answers, but it is not legal, audit, certification, investment or professional advice. Review and adapt the output, and have it checked by the appropriate specialist before you rely on it.
Who can use it?
The page and sample are public. Customising the template is Pro; a free account gets 3 runs in total, shared across every Pro tool, and the generated document is free to view on the page. Download is Pro: downloading or copying the document (Word, PDF, Markdown, CSV) is never part of the free runs. Pro is $5/month or $40/year. Your work is saved in your own browser.
More security tools
The tools people use most on PlayCISO.
Catch injected instructions hiding in content your AI agent should only read.
Score your readiness against the first international standard for AI security threats.
Critical CVEs across 21 major vendors, month by month — and what the 2026 surge means.
Build an ordered ruleset, replay labelled packets and catch shadowed and any-to-any rules.
Self-assess all six Functions, from Govern to Recover, and see your weakest one.
Search 172 threat actors across Microsoft, CrowdStrike, Mandiant and community names.