Skip to content
AI security standards · Template

Training Data Quality & Provenance Checklist

Prove where your training and fine-tuning data came from, that you are allowed to use it, that it is good enough and that nobody poisoned it — with a dataset register that flags anything you should not train on yet.

Template & sample only · See the sample ↓ — a structured starting point with sample content, not legal, audit, certification or investment advice.

Template & sample only. Generated from a PlayCISO template as a starting point: it is not legal, audit, certification, investment or professional advice. Review, adapt and validate it for your organisation before you rely on it.You are looking at sample data. Replace it with your own details below — the document regenerates as you type and your copy is saved in this browser.

For a model you train or fine-tune, prove where every dataset came from, that you are allowed to use it, that it is good enough, and that nobody tampered with it. The checklist follows the data-quality themes of ISO/IEC 5259 and the data life-cycle stages of ISO/IEC 8183; the dataset register records each dataset's source, basis, personal data and checks. The summary flags datasets you should not train on yet.

Also as
Data provenance & quality maturity
54/100
29 of 29 items assessed
Datasets registered
5
533,400 records in total
Datasets with unconfirmed licence or lawful basis
1
Do not train on these until resolved: Medical correspondence sample
Datasets containing personal data (incl. pseudonymised)
3
Historic claims 2019–2025, Handler summary gold set, Medical correspondence sample
Datasets containing special category data
1
Needs specific justification and DPIA: Medical correspondence sample
Bias / representativeness check not complete
3
Historic claims 2019–2025, Medical correspondence sample, Synthetic edge-case claims
No retention or deletion date
1
Medical correspondence sample
Material gaps / red flags (1)
  • Special category and children's data is excluded, or its use is specifically justified and approved

Your context

Dataset register

One row per dataset used for training, fine-tuning or evaluation. Write "Unknown" in the basis column rather than leaving a guess.

Lawful basis & licensing

  • Every dataset has a documented right to be used for model training, and licence terms permit this use (including commercial use and derived models)5259 / 8183material

  • Use of customer or user data for training is permitted by customer contracts and privacy noticesmaterial

  • Web-collected data respects site terms and machine-readable opt-outs from text and data mining, and the collection method is documentedEU AI Act

  • Purchased or third-party datasets come with supplier warranties on provenance and rights42001 A.10

  • The base model's licence and acceptable-use terms permit your fine-tuning and intended use, and its own training-data disclosures were reviewed

Provenance records

  • A provenance record exists per dataset: source, collection method, collection dates, who collected it and every transformation applied8183material

  • Each dataset has a datasheet or data card describing purpose, composition, known gaps and intended uses

  • Synthetic and model-generated data is labelled as such, with the generating model, prompts and filtering recorded

Consent & personal data

  • A lawful basis is identified for each personal-data dataset, and a DPIA covers the training useGDPRmaterial

  • Where consent is the basis, consent records are linkable to the data and withdrawal is handled, including what it means for models already trained

  • Direct identifiers are removed or pseudonymised, and free-text fields are scanned for personal data before trainingmaterial

  • Special category and children's data is excluded, or its use is specifically justified and approvedmaterial

  • The trained model is tested for memorisation and extraction of sensitive training records

Labelling quality

  • Written labelling guidelines with worked examples and edge cases are issued to every labeller

  • Agreement between labellers is measured and disagreements are adjudicated5259

  • Labelling vendors are vetted for security, data access and working conditions, and contracted not to retain data42001 A.10

Bias & representativeness

  • Representativeness is assessed against the population the model will serve, and gaps are documented with their likely effect5259material

  • Features that may act as proxies for protected characteristics (postcode, name, language) are identified and their use justified

  • Data-level measures (class balance, subgroup counts, missing-value rates by group) are recorded per dataset version

Poisoning defences

  • Data is ingested only from authenticated, allow-listed sources, with integrity hashes checked at each stage27090material

  • Write access to training data stores is restricted and logged, and promotion of a new dataset version needs a second approvermaterial

  • Outlier, duplicate and suspicious-pattern screening runs before training, including checks for planted trigger phrases

  • User feedback and other loops that feed back into training are protected against deliberate manipulation

Versioning & lineage

  • Dataset versions are immutable and content-hashed, and every model records exactly which versions trained and evaluated it8183material

  • Preparation pipelines are in version control and can reproduce a training set from raw sources

  • Lineage from raw data through processed data to training set and model is available to auditors on request

Retention & deletion

  • Each dataset has a retention period consistent with its lawful basis, and expiry triggers review or deletion8183material

  • Erasure requests reach training stores, with a documented approach for models already trained on the data (retrain, unlearn or justify)

  • Disposal covers derived copies — caches, feature stores, embeddings, backups, vendor copies — with evidence

Loading…

Sample report · illustrative data

Sample result: training data provenance register

Example register for an invented claims-summarisation model fine-tuned from a foundation model: five datasets, each with its source, lawful basis, personal-data class, bias check and retention date.

Dataset register (excerpt)

DatasetLicence / lawful basisPersonal dataBias checkRetain until
Historic claimsLegitimate interests; privacy notice covers model improvementPseudonymisedPartialEnd of retention period set
Handler summary gold setInternal work productPseudonymisedDoneEnd of retention period set
Medical correspondence sampleTBC, DPO review pendingSpecial category dataNot doneNot set
Public weather eventsOpen government licence (commercial use permitted)NoneDoneWhile model in service
Synthetic edge-case claimsInternal; generator model and prompts recordedNonePartialDate set

Summary figures

Datasets registered
5
Unconfirmed licence or lawful basis
1 (do not train on it until resolved)
Containing personal data, including pseudonymised
3
Containing special category data
1 (needs specific justification and a DPIA)
Bias check not complete
3
No retention or deletion date
1

Checklist items rated (excerpt)

SectionItemStatus
Lawful basis and licensingCustomer data use is permitted by contracts and privacy noticesIn place
Consent and personal dataSpecial category data is excluded or specifically justified and approvedNot in place
Poisoning defencesData is ingested only from allow-listed sources, with hashes checked at each stagePartially in place
Versioning and lineageDataset versions are immutable and content-hashedIn place

Made-up example data. Your own run uses your inputs. Downloads of the full report need Pro.

TL;DR

What this template gives you

4 points
  • Twenty-nine checks across licensing, provenance, personal data, labelling, bias, poisoning, versioning and deletion — for models you train or fine-tune.
  • ISO/IEC 5259 covers data quality for analytics and machine learning; ISO/IEC 8183 describes the stages data passes through in an AI system. The checks follow both, in plain language.
  • A dataset register records source, licence or lawful basis, personal data, version hash, labelling method, bias check and retention per dataset.
  • The summary names datasets with an unconfirmed basis, special category data, missing bias checks or no deletion date.

Frequently asked questions

What is training data provenance?

It is the documented history of a dataset: where it came from, how and when it was collected, who handled it, what was done to it, and on what basis you may use it. Provenance lets you answer a regulator, a customer or a court asking "what was this model trained on, and were you allowed to?" — and lets you find and remove bad data after the fact.

What are ISO/IEC 5259 and ISO/IEC 8183?

ISO/IEC 5259 is a multi-part series on data quality for analytics and machine learning — how to define, measure and manage the quality of data used to build models. ISO/IEC 8183 is a framework describing the life cycle of data in AI systems, from the idea for a dataset through collection, preparation, use and eventual decommissioning. This template draws on both in its own words; it does not reproduce them.

Does this apply if I only use retrieval (RAG) and do not train?

Partly. Licensing, personal data, poisoning defences and retention still apply to the documents you retrieve from; labelling and training-specific bias checks may not. Mark the items that do not apply as N/A.

Is this a finished document I can rely on?

No. Every tool in this suite is a template with sample content. It gives you a structured, editable starting point and generates a document from your own answers, but it is not legal, audit, certification, investment or professional advice. Review and adapt the output, and have it checked by the appropriate specialist before you rely on it.

Who can use it?

The page and sample are public. Customising the template is Pro; a free account gets 3 runs in total, shared across every Pro tool, and the generated document is free to view on the page. Download is Pro: downloading or copying the document (Word, PDF, Markdown, CSV) is never part of the free runs. Pro is $5/month or $40/year. Your work is saved in your own browser.

The tools people use most on PlayCISO.

All tools