Why "works correctly" is now a production incident

Here is a ticket you have written. Not this exact one, but one close enough that it will sting.

Title: Password reset Description: As a user, I want to reset my password so I can get back into my account. Acceptance criteria: Password reset works correctly.

Three words at the bottom. For twenty years that was fine. A developer picked it up, and somewhere between reading it and opening the pull request they did an enormous amount of unbilled work: they remembered that reset tokens should expire, that the token should be single use, that the response should be identical whether or not the email exists so you cannot enumerate accounts, that the old sessions should probably be invalidated, that the email should not contain the password. None of that was in the ticket. All of it arrived anyway, carried in by a person who had been burned before.

Now hand the same ticket to an agent. You will get a password reset. It will work. The token will very likely never expire, because nothing in the ticket said it should. The endpoint will helpfully return “no account with that email” because that is a friendlier error message and friendliness reads as correct. The reset will not invalidate existing sessions, because the ticket said nothing about sessions and sessions are a different concern. Every one of those decisions is defensible against the text you wrote. Together they are an account takeover path, merged on a Thursday, in a pull request that looks clean.

That is the shift. “Works correctly” did not become worse. It became load-bearing.

The gap used to be free

Every acceptance criterion has a gap between what it says and what it means. The measure of a good engineering culture used to be how reliably that gap got filled without anyone talking about it.

The gap was filled by three things. Institutional memory, which is the reason your senior engineer flinches at a certain kind of query. Taste, which is the accumulated sense of what a reasonable implementation looks like in your codebase. And friction, which is the most underrated of the three: writing code was slow enough that the person writing it had time to notice the ticket was thin and go ask.

The friction is what actually held the system together. A vague ticket produced a Slack message. The Slack message produced a clarification. The clarification never made it back into the ticket, which is why your backlog reads like a series of half-remembered conversations, but the software came out roughly right because the conversation happened before the code did.

Agents remove the friction and keep the vagueness. There is no moment where an agent stares at “works correctly”, sighs, and asks what you meant. It resolves the ambiguity instantly, privately, and plausibly, and then it writes two hundred lines that encode that resolution. You do not get a question. You get a diff.

What the agent actually reads

It is worth being precise about the mechanism, because “AI needs better prompts” is not the point being made here.

Your acceptance criteria are not documentation any more. They are the closest thing in the ticket to an executable specification, and they are being consumed as one. Whatever else is in the description gets treated as context; the criteria get treated as the contract. An agent optimises for the contract, because that is what it was asked to satisfy, and because in most workflows the criteria are also what it will be evaluated against.

So the text does three jobs at once now. It tells the agent what to build. It tells a reviewer what to check. And increasingly it tells some automated gate whether the work is finished. A phrase that cannot do the first job will fail the other two as well, silently, because “works correctly” is trivially satisfiable by anything that does not crash.

This is why the failure mode is not “the agent did something stupid.” The failure mode is “the agent did something reasonable, and reasonable was not what you needed, and nothing in your process was capable of noticing the difference.”

The vocabulary of things that cannot fail

There is a specific class of word that shows up in weak criteria. They share a property: they cannot be false. Nothing an agent produces can be demonstrated to violate them, which means they exert no force on the implementation at all.

Works. Works how? Under what input? “The import works” is satisfied by an import that succeeds on a three-row CSV and times out on a real one.

Properly and correctly. These are the purest form. They assert that a standard exists without naming it, and then delegate the standard back to whoever is reading. A human reader delegates it to their own judgement. An agent delegates it to the median of everything it has ever seen.

Handles. “Handles errors gracefully” is the most common untestable criterion in software. Catching an exception and swallowing it handles the error. So does logging it and continuing with corrupt state. Both are graceful in the sense that nothing visibly explodes.

User-friendly, intuitive, clean, seamless. These describe a feeling in a person who is not present.

Fast, performant, scalable, secure. These are the dangerous ones, because they sound technical. They are the same shape as “works correctly” wearing a jacket. “The endpoint should be fast” is satisfied by 900ms if nobody wrote down a number.

Should, in the sense of aspiration rather than requirement. “The job should ideally retry” is not a criterion, it is a preference, and preferences do not survive contact with an implementation.

Etc., and so on, as appropriate, where relevant. These are the ones I find genuinely alarming, because they explicitly hand the boundary of the work to the reader. “Validate the required fields, etc.” is an instruction to guess how much work this is.

If you want a fast diagnostic on your own backlog: search it for these words. Not to be clever, but because the density of them predicts how much of your definition of done currently lives in people’s heads rather than in your system of record.

works        properly      correctly    handles
user-friendly              intuitive    clean
seamless     fast          performant   scalable
secure       should (aspirational)
etc.         and so on     as appropriate
where relevant

Rewriting three of them

Abstract advice about specificity is useless. Here is the actual craft, on three criteria of the kind that are in your backlog right now.

Before:

- Password reset works correctly.

After:

- [ ] POST /auth/reset with a registered email returns 202 and sends
      one email containing a single-use token.
- [ ] POST /auth/reset with an unregistered email returns 202 with an
      identical body and sends no email (no account enumeration).
- [ ] A reset token is rejected after 30 minutes, and rejected on second
      use, both returning 400 with code TOKEN_INVALID.
- [ ] Completing a reset invalidates all existing sessions for that user;
      an in-flight request with an old session cookie returns 401.
- [ ] The reset email body contains no password and no PII beyond the
      user's own display name.

Notice what changed and what did not. It did not get longer because I padded it. It got longer because every ambiguity that a human would have silently resolved is now written down, which means it now applies to an agent, a contractor, a reviewer at 6pm, and you in four months. Notice also that the second criterion is the account enumeration bug from the opening. It exists as a line item now, so it cannot be quietly optimised away in favour of a friendlier error message.

Before:

- The dashboard should load fast even with a lot of data.

After:

- [ ] With a seeded account of 50,000 events, GET /dashboard returns in
      under 800ms at p95 across 20 sequential requests on CI hardware.
- [ ] The endpoint issues at most 4 database queries regardless of event
      count (assert via query counter in the integration test).
- [ ] Events older than the selected range are excluded in SQL, not in
      application code.

The third line is the interesting one. It is not a performance assertion, it is an implementation constraint, and normally I would say criteria should describe outcomes rather than dictate implementation. That advice was written for human developers, who could be trusted to find an equivalent-or-better approach. When the cost of a wrong-but-passing implementation is high and the shape of the right one is already known, writing it down is cheaper than discovering the difference in production. Be sparing with this. It is a tool, not a default.

Before:

- Handle errors gracefully.

After:

- [ ] A 5xx from the payments provider results in the order staying in
      state `pending_payment`, not `failed`, and one retry scheduled at
      60s.
- [ ] A 4xx from the payments provider moves the order to `failed` with
      the provider's error code stored in `orders.failure_code`.
- [ ] No provider error path logs the full card object; the test asserts
      the log line matches the redacted fixture.
- [ ] The user sees the copy in `errors.payment_declined`, not a raw
      provider message.

“Handle errors gracefully” and that block describe the same intent. Only one of them is capable of being wrong, and being capable of being wrong is the entire property you are trying to buy.

The blast radius argument

Someone reasonable will now say: this is just good practice, and good practice has always been good practice. True. The reason it has moved from good practice to urgent is arithmetic.

When one developer worked one ticket over two days, a vague criterion produced one wrong implementation, discovered in review by someone who had context, at a point where changing it was cheap. The vagueness was contained by the slowness.

When several agents work several tickets in an afternoon, the same vagueness produces several wrong implementations simultaneously, all internally consistent, all reviewed by people who now have three times the diff volume and the same number of hours. The reviewer’s ability to fill the gap is exactly the resource that has been consumed. This is why the criteria have to carry the weight: the thing that used to carry it is no longer available in sufficient quantity.

And the wrongness compounds in a specific way. An agent implementing ticket B reads the code produced for ticket A. If ticket A resolved “works correctly” as “no token expiry”, that decision is now precedent sitting in the codebase, and the next agent will match the local style, because matching local style is correct behaviour. Vagueness does not just produce one bad implementation. It produces a convention.

What a machine can and cannot check

There is a practical reason to write criteria as checkboxes beyond readability, which is that a checkbox is the only form of criterion current tooling can evaluate without guessing.

A markdown task list, or a native task list in your tracker, has a binary state that something other than a person can read. If a pull request merges and three boxes are still unticked, that is a fact a machine can act on, and the correct action is to move the ticket to review rather than to done. Prose criteria, including Given/When/Then, do not have that property. They can be read, they can be scored for quality, and with a semantic layer they can be compared against a diff with a confidence estimate, but they cannot be resolved to true or false with certainty. The honest handling of an unverifiable criterion is to route it to a human and say so, rather than to guess and call it done.

That constraint should shape how you write. Not everything can be a checkbox, and pretending otherwise produces absurd criteria. But the split between what can be mechanically confirmed and what genuinely needs a person to look is a split worth making deliberately, at authoring time, rather than discovering it at merge time.

Where this breaks down

I have been arguing that vague criteria are now dangerous. Let me argue the other side properly, because there is a real cost here and pretending there is not would make this advice easy to dismiss.

Specification is not free, and it is not always cheaper than the bug. The password reset block above took me maybe eight minutes to write, and it will take longer for a domain you know less well. On a low-stakes internal tool, eight minutes of specification to prevent a bug that would have taken four minutes to fix after someone noticed is a bad trade. The argument for precision is strongest where the cost of wrong-and-plausible is high: auth, money, permissions, data deletion, anything customer-visible at scale. It is weakest on the CRUD screen three people use. Applying uniform rigour everywhere is how teams end up with a process nobody follows.

Over-specification produces its own failure mode. Criteria that pin down implementation detail get you exactly what you described, including the parts you got wrong. A human developer would have pushed back on “at most 4 queries” if the right answer was 5. An agent will contort the code to hit 4. The more precisely you specify, the more you are betting that your specification is right, and you are no longer able to rely on the implementer’s judgement to catch you. There is a real regime where a thin ticket handed to a strong engineer beats a thick ticket handed to anything, and it is the regime where the engineer knows more than the author does.

Some work genuinely cannot be specified in advance. Research spikes, exploratory refactors, anything where the output is understanding rather than behaviour. “Find out why the queue backs up on Mondays” has no acceptance criteria that are not lies. Teams that mandate testable criteria on every ticket push this work into the shadows, where it gets done anyway with no record at all. Give it a different container and stop pretending.

And the words are a proxy, not the disease. You can strip every instance of “properly” from your backlog and produce criteria that are specific, checkable, and describe the wrong feature. Precision about the wrong thing is worse than vagueness about the right thing, because it is much more convincing. No amount of criteria craft substitutes for knowing what you are building, and I would rather work from “make the reset flow secure, ask me what that means” written by someone who understands the product than from forty crisp checkboxes written by someone who does not.

The takeaway

“Works correctly” was never a specification. It was a pointer to a shared understanding held by the people around the ticket, and it worked because those people were slow enough to be consulted and experienced enough to be right.

Agents removed the slowness and were never party to the understanding. So the pointer has to become the thing. Every ambiguity you leave in a criterion is now a decision you have delegated to a system that will make it instantly, defensibly, and out of your sight.

The practical move is not “write more detail.” It is to write criteria that are capable of being false, and then to be deliberate about which of them a machine can settle and which honestly need a person. That is craft, and it can be taught.

The next piece takes that on directly: how to write acceptance criteria an agent cannot game, including the specific patterns that look rigorous and are trivially satisfiable by an implementation that does nothing useful at all.