The hard scaling problem in AI is deciding what to remember

At small scale, source selection looks trivial. Put execution IDs in a list. Let an operator choose one. Pass it to the replay API.

At production scale, that pattern fails in every direction. The list is too large to load, too stale to trust, too broad to authorize safely, and too expensive to query against a runtime database that was never designed to be a search product.

The interesting question is not “how do we display a million IDs?” It is: what minimum memory does a system need to help a person find the right historical execution without allowing search to become the source of truth?

Search is a product, not a database query

Operators search with incomplete knowledge. They remember a workflow, an approximate date, a customer or service, a status, or a prefix of an execution ID. They need relevance, filters, and a way to narrow the field without waiting for a full table scan.

That calls for a compact execution projection: identity, workflow summary, status, timestamp, replayability signal, and perhaps a short reason when a source is ineligible. It calls for bounded filters and opaque keyset cursors, not offset pagination that gets slower and less stable as data changes.

Kavach’s replay source discovery follows this pattern. The browser asks for a small tenant-scoped page. It can search, filter, and load more. It never receives an unbounded set of execution IDs simply because a form needs a selection control.

A projection is not authority

There is an equally important second half: the search projection cannot be the authority for creating a replay. Projections are optimized for discovery. They can be eventually consistent, intentionally incomplete, and stripped of sensitive payload. They are allowed to be convenient because they are not trusted to decide.

When an operator selects a source, Kavach re-resolves the authoritative full execution before validation or creation. That is where it checks tenant scope, authorization, immutable snapshots, required references, and historical adapter compatibility. A browser selection is a request to inspect—not proof that the execution is still eligible.

Remember less, better

There is a recurring temptation to duplicate everything into the index: complete inputs, outputs, prompts, artifacts, evaluation payloads, and policy documents. It feels efficient until the index becomes a second, poorly governed data warehouse with a larger access surface than the original system.

A better projection stores enough to discover and triage: who owns the execution, what workflow it represents, when it ran, whether it completed, and whether the evidence required for replay appears available. The detailed data remains behind the authoritative resolution boundary, with its own retention, access, and audit controls.

The governance payoff

This separation does more than improve performance. It makes the system’s claims honest. Search results can be stale; the UI can say “available when selected” rather than pretending that a list row proves replayability. If a source is removed, expired, or no longer authorized, creation fails before a new durable Replay is created.

That is a useful pattern well beyond replay. Catalogs, dashboards, lineage views, and recommendation lists should usually be treated as projections. They make complex systems usable. They should not quietly become the place where security or governance decisions are finalized.

The question to ask

When designing an AI control plane, ask two questions for every index and UI list:

• What does a human need to know to find the right thing?
• What must the system re-check before allowing an action that matters?

The space between those questions is where scalable, trustworthy systems are built. It is also where many systems accidentally trade correctness for convenience.