Retrieval is a budget line, not a feature
Indexing everything you own takes an afternoon and is treated as done. It is not done. Every document you leave in the index is charged to every future ticket, in tokens and in wrong answers, and almost nobody has named an owner.
Retrieval gets shipped like a feature. Someone connects the wiki, the tracker, the repository and a few Slack channels, the demo answers a question convincingly, the ticket moves to done, and the team never discusses it again.
Six months later the agent cites a runbook from 2023 as authoritative, and the response is to blame the model.
My position: retrieval is not a feature you ship, it is a standing liability you fund. Index membership is a product decision with a per-ticket cost, it decays without maintenance, and if nobody owns it by name then the answer to “why did the agent think that” is going to be nobody’s job to find out.
The cost is recurring and it is charged to everyone
A feature has a build cost and then it works. That model does not describe an index.
Every document in scope is a candidate for every retrieval, forever. It costs a little in embedding storage, which is trivial and is what people focus on. It costs more in tokens, because retrieved passages get read into the window on tickets that had nothing to do with them.
And it costs most in the thing nobody bills: the probability that a superseded, plausible, well-written document is returned instead of the current one. Similarity is not authority. There is no field in an embedding for “this is the version we agreed on”, and a confidently obsolete answer is more expensive than no answer, because no answer prompts a question and a wrong answer prompts a commit.
That third cost is the one that scales with index size in the wrong direction. Adding a corpus makes recall better and precision worse, and precision is where your incidents come from.
Retrieval as a feature
- Connect every source you have, once
- Success measured by "it found something"
- No owner after the launch ticket closes
- Nothing is ever removed
- Cost modelled as storage, which is nearly free
Retrieval as a budget line
- Sources admitted deliberately, with a reason
- Success measured by what was returned when it mattered
- A named owner and a review cadence
- Expiry is a normal operation
- Cost modelled as tokens plus wrong answers per ticket
Three questions that turn it into a budget line
None of this requires new technology. It requires treating the corpus the way you treat a backlog.
Who owns membership? One person, named, who can say no to a source. Without this, the index only ever grows, because adding a corpus is a five-minute task that always sounds helpful and removing one requires an argument.
What is the expiry policy? Documents need to leave. Not by someone noticing they are stale, which never happens, but by default: a source that has had no edits in eighteen months drops out of scope unless someone renews it. Yes, this will occasionally evict something correct and permanent. That is a smaller cost than keeping four superseded runbooks that read as current.
What gets reviewed, and when? Once a quarter, take the last twenty things the agent got wrong, and for each one, look at what was retrieved. Not what the model concluded, what it was handed. In my experience this is where the surprising findings are, and it is a two-hour exercise that almost nobody runs.
Why this lands on product, not platform
The instinct is to hand the index to whoever runs the platform, because it looks like infrastructure. Storage, embeddings, connectors, latency: all engineering concerns.
But the decisions that matter are not engineering decisions. Which of two conflicting definitions of churn is the real one. Whether last year’s pricing appendix should still be answerable. Whether a discontinued product’s documentation stays in scope for six months or goes today. Those are calls about what your organisation currently means, and the platform team has no basis for making them.
Give a platform team an ownerless corpus and you will get a well-run index of undifferentiated material, because the only lever they can pull without domain authority is more coverage.
Where this breaks down
Curation is slow and the coverage argument is genuinely strong. You can index everything in an afternoon. A governed corpus takes months and it will be incomplete the entire time. For many teams the honest calculation is that a messy index available now beats a clean one available in six months, and I would not argue them out of that. This article is about what to do after the afternoon, not instead of it.
Expiry will evict something you needed. The database migration guide written four years ago that is still exactly right is precisely the document an activity-based policy throws away. Any expiry rule based on edit recency mistakes stability for staleness, and I do not have a clean fix for that beyond a renewal list that someone has to maintain, which is more work than I have made it sound.
Better retrieval systems weaken the whole argument. Recency weighting, source authority scoring and reranking already handle some of this, and they are improving. If the tooling gets good enough at inferring authority, hand-governed membership becomes an expensive way to do something the system does for free. I think authority is a social fact rather than a signal, but that is a belief, not a measurement.
A named owner is a bottleneck by design. The person who can say no to a source is also the person everyone has to wait for. In a large organisation this becomes a queue, and queues get routed around. The federated version, one owner per domain, is more workable and much harder to keep consistent, which is a trade I have not seen anyone solve well.
And the commercial interest is direct. We sell tooling in the specification and context space, so “your index needs governance” is a conclusion that suits us. Discount it accordingly. The part that costs us nothing: the quarterly audit of what was retrieved on failed changes requires no purchase at all, and it is the single highest-value thing in this article.
The takeaway
An index is not a thing you build, it is a thing you keep. Everything left in it is charged to every ticket that follows, mostly in the form of confident answers drawn from documents your organisation stopped believing in.
Name an owner, make expiry routine, and once a quarter read what was actually retrieved on the changes that went wrong.
If you take one thing into next week: find out who in your organisation is allowed to remove a source from the index. If the answer takes more than ten seconds, that is the finding.