← The ADLC library
Live debates · 27

Against prompt libraries

Shared prompt collections look like reusable knowledge and behave like undocumented folklore. They store phrasing rather than intent, they cannot be tested, and they rot without anyone noticing.

Somebody on your team has a document with forty good prompts in it. It started as their own notes, it worked, and the natural next step is obvious to everyone: put it in the wiki, tell the other teams, make it the shared library. Maybe give it a folder in the repository and a review process.

Do not. Or rather, do the small version and stop there, because the thing everyone is imagining is a knowledge asset and what gets built is folklore with version control.

My position: a prompt library is the wrong place to put the knowledge you are trying to keep. The knowledge is in the constraints, not the phrasing, and constraints belong attached to the work and enforced on the output.

What people are actually trying to build

The impulse is right, which is why this is worth arguing about carefully rather than dismissing.

Someone worked out how to get a useful result on a hard, repeated task. That took real effort. The organisation would like the next person not to repeat it. That is a completely reasonable goal and it is the same goal that produced runbooks, style guides and code libraries, all of which worked.

The mistake is in what got extracted. What made the prompt work was almost never the wording. It was that the author knew the repository uses one particular retry helper, that the billing tests are slow so the agent should not be asked to run the full suite, that “user” means three different things in this system and the change concerns only the second. The prompt is a compressed encoding of that knowledge, produced by someone who already held it.

Hand the prompt to someone who does not hold it and they get the compression without the source. It works until it does not, and when it does not they have no way to repair it, because the thing that needs repairing was never written down.

Three properties a library has that this does not

The word library is doing a lot of quiet work, so compare against one.

A prompt collection

  • Correctness is a matter of opinion after the fact
  • No caller list, so nothing can be safely changed
  • Ownership decays to whoever wrote it first
  • Failure looks like a plausible wrong answer
  • Coupled to a model version nobody recorded

A code library

  • Has tests that fail visibly
  • Callers are discoverable, so changes are scoped
  • Ownership is enforced by breakage
  • Failure looks like a stack trace
  • Dependencies are declared and pinned
Every mechanism that keeps a code library honest is absent on the left. What remains is the filing structure.

The one I would press hardest is the middle row. You cannot improve a prompt collection safely, because you cannot find out who is relying on the current behaviour. Someone edits entry 47 to fix a problem in their context, and three teams whose work depended on a clause they never noticed get different output next week with no signal that anything changed.

The rot is invisible, which is the real problem

A stale runbook fails loudly: you follow step four, the command does not exist, you go and find out why. A stale prompt fails quietly. It produces something confident and plausible that is subtly wrong for the current state of the repository.

There are three clocks running against every entry, and none of them are visible in the document.

The model changes. Behaviour that a phrasing was tuned around shifts, sometimes for the better, and the careful hedging that made an entry work becomes noise or, worse, actively misleading.

The repository changes. The helper it names gets deprecated. The convention it encodes gets replaced by a different one that only half the codebase follows yet. Context blindness, output that is internally consistent and incompatible with the system it lands in, is the defining agentic failure, and a stale prompt is a machine for manufacturing it.

The author leaves. Nothing in the document records why any clause is there, so the next maintainer cannot distinguish load-bearing constraints from decoration and will remove the wrong one.

By the time anyone notices, the library has been generating slightly wrong work for months, distributed across enough teams that nobody sees the pattern.

Where the knowledge actually belongs

The useful content of those forty prompts is real. It is just three different kinds of thing wearing one costume, and each has a better home.

What "done" means for this piece of workBelongs in the acceptance criteria on the ticket, written so a machine can settle them. Travels with the work, reviewed with the work, and checkable against the diff afterwards.
How this repository does thingsBelongs in the repository, in the conventions file the agent reads and the linters that enforce it. Applies to every run regardless of who prompted or how.
What must never happenBelongs in a check on the output. A rule expressed as a prompt is a request. The same rule expressed as a merge condition is a constraint.
Each of these survives a model change, an author leaving and a new team adopting the practice. The prompt survives none of them.

There is decent evidence that the effort goes further here too: controlled studies have found human-refined specifications cutting errors by up to half. That improvement comes from precision about the goal, not from precision about the request. Two people can phrase the same criterion completely differently and get equivalent output, which is the clearest sign that the phrasing was never the asset.

What is worth keeping

I am not arguing for nothing. A small, honest version of this is genuinely useful.

Keep a short page of worked examples, dated, with the model named, marked clearly as illustrations rather than approved practice. Ten entries, not two hundred. The purpose is teaching people the shape of a good instruction, not supplying instructions.

The test for whether you have crossed the line: if a new engineer would be criticised for not using the library, it has become policy, and it now needs the ownership, testing and deprecation machinery it structurally cannot have.

Where this breaks down

Repeated, narrow, high-volume tasks are the real counterexample. A team generating the same category of migration fifty times a month genuinely benefits from a fixed, versioned instruction. When the task is stable and the output is checked automatically, the objections mostly dissolve. My argument is strongest for open-ended work and weakest for assembly-line work, and plenty of organisations have more of the latter than they admit.

Criteria and conventions rot too. A repository conventions file goes stale exactly like a prompt does, and I have watched teams carry contradictory rules in one for a year. The difference I would defend is that a convention breaks visibly when a linter fires or a criterion fails, and a prompt breaks invisibly. That is a difference in degree, not in kind.

Not everything fits in acceptance criteria. Some genuinely useful guidance is about approach rather than outcome: explore before editing, ask for the failing case first. That has nowhere else to live and a prompt page is a defensible home for it.

Building the alternative is much harder. Writing criteria a machine can settle is unpleasant, precise work, and it usually falls to someone measured on delivery volume rather than delivery quality. A prompt library takes an afternoon. That asymmetry is the honest reason libraries win, and telling people to do the harder thing is not a plan on its own.

And this can read as purism. Teams are shipping useful work off prompt collections today, and I am arguing against something that is currently helping them. The claim is about where the knowledge sits in three years, not whether the document is useful this quarter.

The takeaway

The instinct to capture hard-won knowledge is correct. The extraction is what goes wrong: what gets stored is the phrasing, and what mattered was the constraint behind it.

Put what done means on the ticket, put how this codebase works in the codebase, and put what must never happen into a check on the output. Then keep a small page of examples for teaching, and refuse to let it become policy.

If you take one thing into next week: pick the three most-used entries in your prompt document and try to write down, in plain sentences, what each one is actually asserting about your system. Whatever you cannot express that way was never knowledge, and whatever you can belongs somewhere better.