Keeping a knowledge base true, the drift problem all over again

Nine articles into a series about why you should write your organisation’s knowledge into a directory of markdown files, it is time to make the strongest argument against doing it.

Here it is. In 2019 a team wrote a beautiful internal wiki. Definitions, architecture, runbooks, the lot. It was genuinely good, and for about a year it was the best thing about onboarding there. Then the team reorganised, two authors left, the architecture changed underneath several pages, and nobody had a job that included fixing it. By 2022 the wiki was a trap. New engineers were told, informally, on their second day, “do not trust the wiki.” Which meant the wiki had achieved a negative value: it cost effort to write, it cost effort to warn people about, and it occasionally caught somebody who had not received the warning.

Now imagine that same corpus with an agent reading it on every task, at speed, without a second day, without a colleague to lean over and say do not trust that page.

That is the real risk of this whole idea, and it is not a footnote. It is the same failure this publication exists to argue about, transposed from the record of work to the record of meaning.

Drift is not a documentation problem

The instinct is to treat staleness as a discipline issue. People should update the docs. If they do not, the culture is bad, or the incentives are wrong, or somebody needs to run a documentation sprint.

That framing has been tried by everybody for thirty years and it has never worked anywhere, which is strong evidence that it is the wrong framing.

Drift is a structural property of any record that is maintained separately from the thing it describes. The record is a copy. The original changes. Keeping the copy aligned requires an act of human attention that produces no immediate value to the person performing it, and any process built on repeated attention with deferred payoff loses to any process built on attention with immediate payoff. Every time, in every organisation, regardless of culture.

This is exactly the argument for reading ground truth from source control rather than asking people to move tickets. The ticket is a copy of what the code is doing. The board drifts not because engineers are lazy but because updating a copy is unrewarded work, and unrewarded work is the first thing that goes when a sprint gets tight.

A knowledge base is a copy too. tables/subscription-events.md is a copy of facts about a table. behaviour/grace-period.md is a copy of a rule implemented in code. Every concept file is a statement about a world that keeps moving.

The board drifts because

• The ticket is a copy of the work
• Updating it is unrewarded
• Nothing detects divergence
• Wrongness is invisible until a report
• The fix is asking people to try harder

The knowledge base drifts because

• The concept is a copy of a fact
• Updating it is unrewarded
• Nothing detects divergence
• Wrongness is invisible until an incident
• The fix is asking people to try harder

I have deliberately made both sides of that figure the failure state, because the comparison is not between a good option and a bad one. It is the same disease in two organs.

Why staleness is worse than absence

The claim I keep making, and should now justify properly, is that a stale knowledge base is worse than no knowledge base. It sounds like rhetoric. It is arithmetic.

Consider an agent that needs to know the grace period for a failed payment.

With no knowledge base, it reads the code, or it infers, or it asks. Its output carries the epistemic status of a guess, and everybody downstream treats it accordingly. The reviewer looks at the number and thinks, where did that come from, and checks.

With a stale knowledge base, the agent reads behaviour/grace-period.md, which says fourteen days, and which was true until a change last spring made it seven for annual plans. The agent produces code implementing fourteen days, in a pull request that cites an internal document. The reviewer sees a citation to a reviewed internal reference and checks less carefully, because a citation is a signal of diligence.

The knowledge base has not merely failed to help. It has consumed the reviewer’s scepticism, which was the last defence.

AbsentOutput reads as a guess and gets checked

StaleOutput reads as sourced and gets waved through

TrueOutput is right and the check is quick

This is the same reason a board that is confidently wrong is worse than a board everybody knows is unreliable. Uncertainty that is visible gets managed. Uncertainty that is disguised as certainty gets acted on.

What actually keeps a record true

If discipline does not work, what does? Three mechanisms, in descending order of reliability, and it is worth being clear that only the first is really robust.

Derivation. The record is generated from the thing it describes, so it cannot be wrong for longer than the generation interval. A table document whose column list is generated from the schema is never stale about columns. This is the gold standard and it applies to a smaller share of knowledge than anybody wants, because the valuable parts (meaning, intent, exclusions, history) are precisely the parts that cannot be derived.

Coupling. The record is not generated, but changing the underlying thing forces a decision about the record. If touching the billing service surfaces the four concept files that reference it, and the pull request cannot merge until somebody says whether they are still accurate, then maintenance happens at the moment of maximum context, performed by the person who knows the answer. This is the mechanism with the best cost-to-value ratio for the knowledge that cannot be derived, and it is where I would put the engineering effort.

Detection. The record is checked against reality by something automated, and divergence produces a signal. Link checking is the trivial case. Cross-checking a stated constant against the code is a harder case. Detection does not fix anything, but it converts silent drift into a visible queue, and a visible queue can at least be triaged.

Below those three there is a fourth thing that is not a mechanism at all: periodic review, which is the industry standard and which fails because a review with no consequence for skipping is a review that gets skipped, and a date bumped without reading is worse than no date.

Derive itCannot drift beyond the generation interval. Applies to inventory, not to meaning.

Couple it to the changeTouching the code surfaces the concepts that reference it. Maintenance at the moment of maximum context.

Detect divergenceBroken links, contradicted constants, orphaned concepts. Does not fix, but makes the queue visible.

Mark what is unverifiedIf none of the above applies, say so in the document. Honest uncertainty beats disguised confidence.

That last band deserves more attention than it gets. If a concept cannot be derived, coupled or checked, the correct response is not to refuse to write it. It is to write it and say plainly, in the document, that this is a human assertion with no verification path. An agent reading that treats it differently, a reviewer reading it treats it differently, and you have preserved the scepticism that a bare confident statement destroys.

The agent-updates-its-own-knowledge question

The LLM-wiki framing includes maintenance by the model, and it is the most attractive and most dangerous idea in this whole space, so it deserves a straight answer.

The attraction is obvious. Coupling, described above, requires somebody to notice that a concept file has gone out of date. An agent working on the change is well placed to notice, and perfectly capable of proposing an edit. Maintenance becomes a side effect of the work rather than a separate chore. That is exactly the shape of solution that historically beats discipline.

The danger is equally obvious. You now have a system that writes plausible statements into your source of truth, faster than anybody reads them, and prose diffs are harder to review than code diffs because there is no test that fails when a sentence becomes subtly untrue.

The resolution is not novel and it is not optional: the same gate you would demand for any automated change. A proposed knowledge edit is a pull request, not a write. It is attributable to the agent and the task that produced it. It is reviewable by whoever owns that part of the directory. It is reversible. And the review is a real review, which means the volume has to stay low enough that reviewing is possible, which in turn means agents should propose edits when they have direct evidence of divergence and not otherwise.

This is precisely the position GroundTruth takes about automated status: writing to the record is fine, writing to the record without attribution, verification and a reversible trail is not automation but abdication. That argument was made about tickets. It transfers to knowledge without modification, and if anything the stakes are higher, because a wrong ticket misleads a standup and a wrong definition misleads every future change.

Where this breaks down

This is an article about the limits of the series, so this section is about the limits of the limits.

The counsel of perfection produces nothing. Everything above can be read as “do not write anything down unless you can verify it,” which would leave you with an empty directory and an organisation that knows nothing collectively. That is worse than a slightly stale wiki, and I do not want to argue anybody into paralysis. Most knowledge is not load-bearing, most staleness is survivable, and a mediocre knowledge base that people use is more valuable than a perfect one that does not exist. The strong warnings apply to the small set of concepts where being wrong is expensive.

Coupling is harder to build than a paragraph makes it sound. Surfacing the right concept files when a piece of code changes requires knowing which concepts relate to which code, and that mapping is itself a record that can drift. You can approximate it with paths, links and heuristics, and the approximation will be wrong in both directions: prompting for irrelevant files, missing relevant ones. Prompt-fatigue is a real risk, and a coupling check that fires too often gets disabled, at which point you are back to discipline.

Nothing here is specified by the format, and the format is v0.1. There is no verification state, no freshness signal, no supersession, no ownership. Every mechanism in this article is a convention you build on top of a spec published in June 2026 that requires one field. If somebody tells you their knowledge base is trustworthy because it is OKF-conformant, they have misunderstood what the spec claims.

And the honest bottom line: maintenance is the whole game, and most teams will not do it. Nine articles of structural argument do not change the base rate. Most knowledge bases rot. If you take one thing from this series, take the sizing decision: curate the small set you will genuinely maintain, derive everything derivable, mark plainly what is unverified, and resist the pull to document everything, because the failure of a knowledge base is not usually that it was badly designed. It is that it was too big to keep true.

The takeaway

A knowledge base drifts for the same reason a board drifts: it is a copy, maintained by human attention, with no mechanism that makes divergence visible or expensive. Believing that better discipline will fix it in one case while accepting that it never has in the other is not a coherent position.

Staleness is not a weaker version of accuracy. It is worse than absence, because it converts a guess (which gets checked) into a citation (which does not). The mechanisms that actually work are derivation where possible, coupling to the moment of change where not, detection to make drift visible, and honest marking of what is unverified. Agent-authored updates are promising and require the same gate as any other automated write: attributable, reviewable, reversible.

That is the end of this series. The Open Knowledge Format is a small, early, vendor-neutral proposal that knowledge should be a directory of markdown files, one concept per file, path as identity, linked together, with one required field. Its shape is right. Its maintenance story is unwritten, and that is the part that decides whether any of it works.

If you want the general form of the argument rather than the knowledge-specific one, read the piece on ground truth as a principle, which makes the case that every record which cannot maintain itself from something definitionally true will eventually lie to you.