Curated context versus retrieval: when structure beats search
You can index everything you already have in an afternoon. Curating a knowledge base takes months. The case for doing the slow thing is narrower than advocates claim and stronger than sceptics think.
The objection arrives about ninety seconds into any conversation about curating a knowledge base, and it is a good objection.
“We already have all of this. It is in Confluence, in Slack, in the code, in twelve thousand closed tickets. Embed it, index it, let the agent search. Why would we spend six months writing markdown files that duplicate what we already wrote?”
The honest answer is that quite often you should not. Retrieval over existing material is cheap, fast to set up, covers everything, and requires no behaviour change from anybody. Curation is slow, expensive, partial, and demands that busy people write things down. If you compare them as alternatives, retrieval wins on every axis that a spreadsheet can measure.
They are not really alternatives, and the interesting question is not which is better but which failure mode you can live with.
Two different failure modes
Retrieval fails by returning something plausible and unauthoritative.
Ask a semantic index what churn means and it will return the four passages most similar to your question. One is a Confluence page from 2024. One is a Slack message where somebody was thinking out loud. One is a comment in a dbt model. One is a paragraph from a deck. All four are about churn. Three of them are wrong or superseded. The index has no concept of authority, because similarity is not authority, and there is no field in an embedding for “this is the one we agreed on.”
The model then does what models do with conflicting sources: it produces a synthesis. A confident, fluent, reasonable-sounding blend of three wrong things and one right thing. And critically, the output looks identical whether the sources agreed or not.
Curation fails by being incomplete.
A curated knowledge base contains the two hundred things somebody bothered to write. It does not contain the ten thousand things nobody did. When an agent asks about something outside the curated set, it gets nothing, and has to fall back on the code, on inference, or on invention.
Retrieval fails by returning noise as signal
- Similarity is not authority
- Superseded material ranks as well as current
- Conflicts get blended, not flagged
- Output looks the same either way
- You cannot tell which answer was used
Curation fails by having a gap
- Coverage is whatever somebody wrote
- Outside the set, you get nothing
- Absence is at least detectable
- What is there is reviewed and attributable
- You can see exactly which file was read
That asymmetry is the core of the argument. A gap is a bug you can find. A confident synthesis of contradictory sources is a bug you find in a board meeting.
The specific thing curation provides
Strip away the advocacy and curation gives you exactly one thing that retrieval structurally cannot: a designated answer.
When there is one file at metrics/churn-rate.md and its content has been reviewed, the question “what does churn mean here” has an answer that somebody stands behind. That is not a retrieval property. You cannot get it by indexing harder, because the corpus you are indexing contains no signal about which of its statements is authoritative. The authority has to be created by a human act of designation, and designation is what curation is.
Everything else people claim for curation is secondary or achievable other ways. Better chunking? Marginal. Higher-quality prose? Sometimes. Structured traversal? Nice, and partially replicable with a good index. The designated answer is the whole prize, and there is no shortcut to it.
This is also why the file path matters so much. The path is what makes designation stable. metrics/churn-rate.md is a promise that this is where the answer lives, and that promise is what lets a reader stop looking. Retrieval can never let you stop looking, because there is always a fifth result.
Retrieval answers "what does the corpus say about this." Curation answers "what is true here." Those are different questions, and only one of them can be asked of an index.
Where each one actually wins
So the practical position is a split, and the split is not fifty-fifty. It runs along a line that is easy to state.
Curate what must be right. Definitions the business argues about. Behavioural rules. Policies with legal or financial consequence. Constraints that are expensive to violate. The set of things where a plausible wrong answer causes real damage is usually surprisingly small, in the low hundreds of concepts for a mid-sized company, and it is dominated by things people already argue about repeatedly.
Retrieve everything else. Historical context, prior discussions, how something was done before, what was tried, examples, incident postmortems, the vast archive of material that is useful when relevant and harmless when slightly off. This is the long tail, it is enormous, and nobody is ever going to curate it.
Prefer derivation over both, when the fact is derivable. If the answer is mechanically obtainable from code, schema, configuration or source control, get it from there. This is the category people forget, and it is the largest one. A hand-written document describing which services call which is a copy of a fact that already exists in the code, and copies drift. Curate meaning; derive inventory; retrieve history.
Curate
Meaning. Definitions, behavioural rules, policies, constraints. Small set, high consequence, needs a designated answer.
Derive
Inventory. Schemas, dependencies, ownership, what calls what. Already true somewhere machine-readable. Never hand-maintain it.
Retrieve
History. Discussions, postmortems, prior art, examples. Huge, low consequence per item, nobody will ever curate it.
Leave alone
Judgement and situated context. Resists all three. Pretending otherwise produces confident, bland, wrong documents.
Structure makes retrieval better anyway
There is a further point that gets lost when these are framed as rivals: a curated directory is not a competitor to your index. It is the highest-quality input your index will ever get.
A well-cut concept file is a natural retrieval unit. It is about exactly one thing, so its embedding is not an average of four unrelated topics. It is self-contained, so retrieving it does not require assembling fragments. It has front matter that says what kind of thing it is, so results can be filtered by type rather than ranked purely by similarity. And because the path is the identity, a citation to it is checkable by a human in one click.
Compare that with chunking a fifty-page PRD, where the chunk boundaries fall mid-argument, every chunk carries the ambient topic of the document rather than its own subject, and a citation points at “page 31” of something written for a meeting two years ago.
So the sequencing that works is not “choose one.” It is: index everything now because it is cheap and it helps immediately, curate the small high-consequence set over the following months, make the curated set win when it disagrees, and keep deriving whatever is derivable.
That second step is the one I would emphasise most. Do not decide up front what to curate. Let the failures tell you. Every time an agent produces something wrong because it did not know a fact, that fact is a candidate concept, and the list generated this way is dramatically better prioritised than any list generated by a workshop.
Where this breaks down
The curated set is small and the boundary is invisible to the agent. An agent has no reliable way to know whether the absence of a concept file means “this is not written down” or “this is not important.” It cannot tell the edge of the curated region from the edge of the world. So the theoretical advantage of curation (a designated answer) only helps for questions inside the set, and outside it you are back to retrieval with an added risk: the agent may have inferred from the presence of a knowledge base that anything unlisted is unconstrained.
Designation without maintenance is just a more persuasive form of noise. Everything in this article that praises the designated answer assumes the answer is current. A curated file that is eighteen months stale outranks a correct Slack message from last week, because you told the system that curated wins. You have built a mechanism that promotes your stalest confident statement over your freshest accurate one. This is the single strongest argument against curating anything you are not prepared to maintain, and it is why the next article in this series is entirely about drift.
Retrieval keeps improving and some of this argument may erode. Better indexes, better re-ranking, better handling of recency and conflict: none of these give you designation, but they narrow the gap on everything else, and I would expect several of the practical advantages I have claimed for structure to look weaker in two years. The part I do not expect to erode is authority, because authority is a social fact rather than a technical one.
The v0.1 format offers no help with the routing decision. Nothing in the spec says what to curate, how to mark a concept as authoritative over an indexed source, or how a consumer should resolve a conflict between them. That is all convention, all yours to build, and it is the actual hard part of the split described above.
And the cost of curation is chronically underestimated by people who like writing. Six months of curation effort is a real programme with a real opportunity cost. The first thirty files are enjoyable and take a week. The organisational work of getting definitions agreed, owned and kept true is the other ninety-five percent, and it never finishes. If the honest answer is that your team will not sustain it, then index everything, derive what you can, and do not build a monument you will have to apologise for later.
The takeaway
Retrieval and curation fail differently, and that difference is the whole decision. Retrieval fails by blending superseded material into confident answers you cannot audit. Curation fails by having gaps you can see. For anything where being wrong is expensive, a visible gap is the better failure.
Curation buys exactly one thing that no amount of indexing provides: a designated answer at a stable path that somebody stands behind. Spend that on meaning, derive inventory from the artefacts that already contain it, and let retrieval cover the long tail of history.
And prioritise by watching what breaks rather than by planning a documentation programme, because the concepts that keep producing wrong answers will name themselves faster than any workshop will.
The final piece in this series confronts the objection running underneath all nine of the others: a knowledge base that goes stale is worse than no knowledge base at all, and staleness is the same drift problem the record of work already has.