Quick Take
- Data provenance is proof of where your data came from and how it was used.
- The EU AI Act will require this proof for high-risk AI systems from December 2027.
- The hard part is proving it for data you don’t fully own or control.
- The fix pairs documentation at each data source with a record of which datasets a model used.
Ask a regulator today how they know an AI system’s output is trustworthy, and the answer comes down to the data. Where it came from, whether you had the right to use it, and whether you can prove any of that on demand.
That question used to be optional. It isn’t anymore.
What is data provenance?
The data provenance meaning is straightforward: it’s the documented history of a piece of data. Its origin, who created or collected it, what license or consent governs it, and every transformation it went through before it reached a model. It answers one question a court, a regulator, or an auditor will eventually ask: how did this data end up here, and can you prove it?
Provenance is often confused with data governance more broadly. Governance is the full set of rules for how data is managed. Provenance is narrower. It’s the evidence trail that makes those rules verifiable after the fact.
Did you know?
An audit by MIT’s Data Provenance Initiative, which covered more than 1,800 fine-tuning datasets used to train popular models, found license information was miscategorized in over half of them, with terms omitted entirely more than 70% of the time. Many organizations training on public or third-party data don’t actually know what they’re allowed to do with it.
Data provenance vs data lineage
The two terms get used interchangeably, but they answer different questions, and the EU AI Act asks for both.
| Data provenance | Data lineage | |
| Answers | Where did this come from, and was I allowed to use it? | What technical path did this data take through my systems? |
| Focus | Origin, rights, consent, authenticity | Pipelines, transformations, processing steps |
| Owned by | Legal, compliance, data governance | Data engineering |
| Regulatory relevance | Required for high-risk AI from December 2027 (EU AI Act Article 10, Annex IV) | Also asked for: how data was labeled, cleaned, and processed (Annex IV) |
Lineage tells you how data moved through your pipeline. Provenance tells you whether it should have been there at all. You need both, and the AI Act asks for both: where the data came from, and how it was labeled and cleaned.
Why AI regulation will require it from December 2027
The EU AI Act is the clearest example of provenance moving from best practice to legal obligation. Its EU AI Act training data governance rules, set out in Article 10 , will require that training, validation, and testing datasets for high-risk AI systems be subject to data governance practices covering their collection, origin, and any preprocessing.
The datasets will have to be relevant, sufficiently representative, and, as far as possible, free of errors.
Regulation (EU) 2026/1744, in force since July 2026, moved these obligations to 2 December 2027 for high-risk systems in the Annex III use cases, and to 2 August 2028 for high-risk AI built into regulated products.
Annex IV is where that requirement becomes concrete. Section 2(d) will require providers to describe the datasets used, including their origin, scope, and main characteristics.
The Annex also asks how the data was obtained and selected, how it was labeled, and how it was cleaned.
Generic statements don’t satisfy the requirement. Auditors expect actual dataset descriptions and a real account of where the data came from.
Part of the AI Act already applies. Since August 2025, Article 53 has required providers of general-purpose AI models to publish a summary of the content used to train them.
This isn’t isolated to the EU. US state laws are moving in a similar direction, even where the language differs. California’s AB 2013, in force since January 1, 2026, requires developers of generative AI systems to publish a summary of the data used to train them, including its sources and owners and whether it contains personal information.
Colorado’s SB 26-189, which replaced the state’s original AI law and takes effect on January 1, 2027, drops impact assessments but still requires developers to give deployers documentation about AI used in consequential decisions. Different mechanisms, same underlying ask: show your work.
The push isn’t only regulatory, either. A joint security advisory from the NSA, CISA, and the FBI names provenance tracking as a core defense against data poisoning, where attackers manipulate training data to skew a model’s outputs. An unverifiable dataset is as much a security gap as a compliance one.
Pro tip:
If your technical documentation describes a dataset as “internal customer data” with no further detail, that won’t pass an Annex IV review. Regulators expect the origin, the legal basis for use, and the preprocessing steps, per dataset, not per system.
Training data documentation: what regulators expect to see
A dataset description that satisfies Article 10 and Annex IV generally needs to cover:
• Source: Where the data originated, and whether it was collected first-party, licensed, or obtained from a third party.
• Legal basis: The license, consent, or contractual right that permits its use for this purpose.
• Collection and preprocessing: How the data was gathered, cleaned, labeled, and transformed before training.
• Timestamp and versioning: When the dataset was created and how it has changed since.
• Known limitations: Gaps in representativeness or quality that could affect outputs.
None of this is difficult to produce for data your organization fully owns and controls. It gets much harder the moment the data doesn’t.
Why provenance breaks down across separate organizations
Here’s where many provenance strategies hit a wall. You can document the origin of a dataset sitting in your own warehouse. You cannot document, in the same way, a dataset that belongs to a hospital network, a bank partner, or a government agency you’re collaborating with, because you were never allowed to copy or centralize it to begin with.
This is the actual bottleneck behind many “we can’t prove provenance” conversations. Teams understand the requirement well enough. The standard way of meeting it, pulling data into one place so you can document and audit it centrally, simply isn’t available when the data is sensitive, regulated, or owned by someone else.
Federated learning and other privacy-preserving approaches change this by keeping data at its source. The model moves to the data instead of the data moving to the model.
Each organization documents where its data came from and the rights attached to it. The platform then records which datasets and computations a model used.
Duality’s take
Many provenance gaps trace back to copies, because every copy is another version to track with its own terms of use. Keeping data where it lives avoids those copies, but storage location says nothing about where data came from or who may use it. Origin and rights still have to be documented at each source.
This is also why data sovereignty and provenance often come up together: one covers where data may be stored and processed, the other covers where it came from and who may use it.
Provenance tracking across the AI model lifecycle
Provenance isn’t a one-time checkbox at training time. It needs to hold up across the full lifecycle:
1. Data collection: Capture source, license, and consent at the point of ingestion, not retroactively.
2. Training and fine-tuning: Record which dataset version, in which state, trained which model version.
3. Deployment: Maintain the link between a live model and the training data that produced it, so a regulator can trace back from output to origin.
4. Post-market monitoring: Update documentation as data or models change, since the EU AI Act treats this as a living requirement, not a one-time filing.
Our experts’ advice:
Start with your highest-risk datasets, the ones tied to regulated decisions or cross-border collaboration, and build a repeatable provenance record for those first. Trying to retrofit documentation across every dataset at once is how many provenance projects stall.
What “verifiable” data sources actually means
A source is verifiable when its origin can be traced back to something independent of your own say-so: a primary record, a signed license, a documented chain of custody, or a computation log that shows exactly what ran and on what.
Self-reported metadata (a spreadsheet saying “collected in 2024, internal use only”) isn’t verifiable on its own. It becomes verifiable when it’s backed by an audit trail that a third party could check without having to trust your account of it.
This is the standard auditors often apply, and it’s also the standard that separates organizations that can move fast on cross-organization AI projects from those stuck waiting on manual reviews every time a new dataset enters the picture.
Where this hits hardest
Provenance gaps aren’t evenly distributed. They show up often in sectors where cross-organization data collaboration is valuable and restricted.
- Financial services firms trying to build shared fraud or anti-money-laundering models across institutions run into this immediately. Each bank can document its own transaction data. None of them can centralize a competitor’s or partner’s data just to prove where a joint model’s training set came from.
- Healthcare research faces the same wall. A multi-hospital study needs patient data from every participating site, and HIPAA allows research use only under specific conditions. A pilot with NHS England, the US National Cancer Institute, and the UK’s Department for Science, Innovation and Technology (DSIT), using Duality’s platform, cut approval time for an international collaboration from over a year to about two months.
- Government and defense agencies deal with a version of this that’s compounded by classification and jurisdiction. Data from one agency or allied partner often can’t move at all, which makes traditional provenance documentation, built around a copy you can inspect, structurally impossible to produce.
In each case, what helps is a setup that works without pooling the raw records and keeps a record of which datasets and computations a model used.
Getting ahead of this before the audit
Waiting for a regulator to ask the question is the expensive way to find out where your provenance gaps are. A few starting points:
• Inventory which of your AI systems touch data you don’t fully own or control.
• Check whether your documentation would meet Annex IV Section 2 for each dataset before December 2027.
• Look at where centralizing data has become a workaround for proving provenance, and whether that workaround is itself a compliance risk.
• Build provenance capture into new data collaborations from day one, rather than adding it after a project is already running.
How Duality helps
Duality’s platform helps organizations work with sensitive data without pooling the raw records. Government agencies, hospitals, and financial institutions can collaborate across borders, partners, and jurisdictions, with a record of which datasets and computations a model used.
Teams can add that record to the documentation each organization keeps on where its data came from and the rights attached to it.
