The incident report is not evidence

Most teams discover their evidence model at the worst possible time: after a surprising outcome, when someone asks for an explanation and the answer is scattered across dashboards, traces, prompt histories, feature flags, and the memories of the people who were on call.

That is an incident report. It may be useful. It is not the same thing as evidence.

An incident report is reconstructed after the fact. It is selective, narrative, and optimized for communicating what a team now believes happened. Evidence is what allows that belief to be challenged. It is the durable set of identifiers, inputs, configurations, evaluations, policies, jobs, and decisions that make a claim independently inspectable.

The comfort of observability

Observability is indispensable. Metrics reveal trends, traces reveal timing, and logs reveal local context. The problem begins when teams mistake visibility for accountability.

A dashboard can show that answer quality fell last Tuesday. It usually cannot answer whether the evaluation used the same rubric as last week, whether the model alias moved, whether the policy version changed, or whether a retry produced the output being reviewed. A trace can capture a tool call, but not necessarily the governed relationship between that call and the decision it informed.

Evidence is relational

The important unit is not a giant JSON blob. It is a set of stable relationships. An evaluation belongs to an execution. A decision refers to the policy version that shaped it. An asynchronous job records who submitted it and under which request and correlation context. A replay result points to the baseline and candidate evaluations used to compare historical and reproduced behavior.

Those relationships let an operator move from a claim to the artifacts that support it without inventing the graph in the browser or in a post-incident spreadsheet. They also let the system state its limits. A missing baseline, unavailable artifact, or failed projection is visible as a missing link—not silently filled with inference.

Audit is not a graveyard

Audit systems often become write-only graveyards: lots of events, little ability to answer a question. A useful audit record needs a job or resource identity, an operation, actor and tenant context, timestamps, outcome, and enough correlation metadata to connect it to the governed work.

That does not mean storing every secret, prompt, or raw artifact in every audit event. In fact, duplicating sensitive data widely is a poor governance strategy. It means recording safe, durable references and enforcing a clear boundary around where fuller evidence lives.

The test

There is a simple test for whether a system has evidence rather than merely logs. Give an operator a decision ID, a job ID, or an execution ID and ask them to answer these questions without opening a terminal or asking the original developer:

• What was requested, by whom, and in which tenant/project?
• What configuration, policy, and evaluator versions were in force?
• What execution and evaluation artifacts support the outcome?
• Which evidence was compared, and what changed?
• What is known, and what is explicitly unavailable?

If the answer requires joining five systems by timestamp and intuition, the operational architecture is carrying more trust than it can justify.

Make review native

Kavach treats review as a first-class workload. Jobs are durable; identities are idempotent; evaluation history is retained; replay lineage and comparison references are persisted; Studio renders backend-owned read models rather than manufacturing governance conclusions in React.

The deeper point is broader than any one implementation. Systems that make decisions should be built to receive questions. Not just “is it up?” or “what is the error rate?” but “why did this outcome deserve to be trusted?”