Digital Provenance Is Trust Calibration Infrastructure

I kept noticing the same pattern across different cases this semester. The Air Canada chatbot that invented a refund policy in February 2024. The enterprise team that abandoned an AI tool after two hallucinations in a demo. The organization that could not tell which of its internal systems depended on a compromised open-source library. Each case looked like a different problem: bad deployment, user negligence, supply chain blind spot. But the underlying mechanism was the same. None of these people had verifiable information about what the system was, where its components came from, or whether its output should be trusted in that specific context. They were calibrating trust on surface signals: tone, fluency, organizational branding, a demo that looked good, the name of a vendor. And surface signals are not a reliable basis for trust calibration.

Lee and See (2004) define trust calibration as the alignment between a person's perception of a system's capability and the system's actual capability. When the two match, you get calibrated trust: relying on the system when it is reliable, overriding it when it is not. When they do not match, you get overtrust, which they call misuse, relying on automation beyond its actual capability, or undertrust, which they call disuse, rejecting automation that could improve performance. The goal is not maximum trust but accurate trust. And accurate trust requires information about the system that the user can verify against something other than the system's own presentation.

The problem is that most users do not have access to that kind of information. When a person interacts with an AI system, they make judgments based on what they can observe: the interface design, the brand, how fluent the output is, whether the recommendation sounds reasonable. These are the same cues that drove the Air Canada passenger to trust the chatbot's invented refund policy. The chatbot spoke with the airline's voice and used the right vocabulary. From the passenger's perspective, it looked and sounded like the airline. There was no structural signal that said the system was improvising beyond its actual authority. Calibration was impossible because the information needed to calibrate was not available at the point of interaction.

Digital provenance solves this by embedding verifiable information about a system's origin, composition, and integrity into the artifact itself. Three mechanisms are converging. Bills of materials, which list every component in a software system including versions and known vulnerabilities. Attestation databases, which cryptographically record who built a given component and when. Watermarking, which marks AI-generated content so users can distinguish synthetic output from human-produced material. Together, these mechanisms give users information that does not depend on what the system says about itself. You do not have to trust the chatbot's claim about its capabilities. You can check the bill of materials to see which model version it runs, what data it was trained on, and what safety evaluations it passed. That check is structural, not relational.

Consider software bills of materials, SBOMs. When a widely used open-source library is hit by a supply chain attack, every organization that depends on that library needs to know whether their systems are affected. Without an SBOM, the process involves calling vendors and hoping someone remembers. With an SBOM, the attack surface is mapped immediately because the component list is machine-readable and traceable across the entire software stack. This is trust calibration at organizational scale. The SBOM does not make the library secure. It gives the organization information to calibrate how much risk they are accepting, which systems need patching first, and which vendors were exposed. The same logic extends to model bills of materials for AI. An MLBOM tells you what training data was used, what fine-tuning was applied, what benchmarks the model passed, and what known limitations exist. If you are deploying a model in a regulated or high-stakes setting, that information is the difference between informed delegation and blind reliance.

This is where I think the regulatory picture snaps into focus. Recent industry analysis points to growing mandates for watermarking and provenance tracking, and the EU AI Act is the most visible implementation. Its watermarking requirement for AI-generated content forces deployers to mark synthetic output so users have a structural basis for distinguishing what a model produced from what a human wrote. The law is saying, in effect, that users cannot calibrate trust in AI output without infrastructure-level information about what they are looking at. A watermark is not a trust mechanism. It is a calibration mechanism. It does not tell you whether the content is accurate. It tells you whether the content was generated by an AI, which is a precondition for deciding how much scrutiny to apply to the output. The regulation recognizes that calibration cannot happen if the user cannot tell what kind of thing they are evaluating.

McKnight et al. (2002) would recognize digital provenance as institution-based trust made operational. Institution-based trust is the belief that protective structures, guarantees, and regulations create safety for dependence on a specific party. Digital provenance infrastructure is exactly that: structural guarantees that operate across systems, maintained by standards bodies, regulators, and platform providers. A user does not need to evaluate each system's trustworthiness from scratch. The infrastructure provides a baseline of verifiable information that applies wherever the provenance mechanism is present. This is institution-based trust, not in the abstract sense of believing that regulations exist, but in the concrete sense of having machine-readable evidence that a system has been attested, its components are documented, and its output is marked.

The piece I keep coming back to is that digital provenance is the first trust calibration mechanism in information systems that works at infrastructure level rather than requiring users to individually evaluate each system. Every trust mechanism before it, from brand reputation to interface design cues to user reviews to vendor certification, placed the calibration burden on the user. You had to assess each system separately, drawing on whatever information you could gather. Provenance infrastructure moves calibration from a cognitive task performed by each person at each interaction to an information environment that is embedded in the artifact. The infrastructure does not judge trustworthiness for you. It provides the verifiable information you need to judge it yourself, without depending on how the system presents itself. That shift, from relational to structural, from individual to infrastructural, is what makes digital provenance different from every trust mechanism that came before it. I wrote about why calibrating trust is structurally different from measuring trustworthiness, and about how AI hallucination is a calibration problem rather than a retrieval problem. Provenance connects both threads. If calibration requires information about what a system is and what it can actually do, provenance is the infrastructure that supplies that information.

The question is no longer whether you trust the system. The question is whether you have the infrastructure to know what you are trusting.