When to say "needs human verification" and mean it
A decline is only useful if it changes what somebody does. Most teams turn theirs into a checkbox within a fortnight, and then wonder why the gate stopped catching anything.
Two weeks after a team turned on criteria checking, I watched their pull request template do something quietly fatal. Of the eleven acceptance criteria attached to the change, four came back verified, one came back contradicted, and six came back “needs human verification.”
The reviewer read the contradicted one, went to the diff, found the problem, and fixed it. Good. Then he looked at the six declines, thought for about four seconds, ticked the box that said “human verified,” and merged.
He was not being lazy. He had no idea what he was supposed to do with them. The system had told him it did not know something, without telling him what it did not know, why it could not tell, or what evidence would settle it. Faced with six shrugs and a merge button, he did the only thing available.
Within a month that team’s human-verified rate was effectively one hundred percent of declines, and their gate had become a slower way to merge exactly what they would have merged anyway.
This is the piece the previous one promised. Declining to answer is the single most valuable behaviour a verification layer has. It is also the easiest to render meaningless, and the failure is not in the tool.
The decline is the honest part
Start from why the decline exists at all.
A verification layer reads a change and a set of criteria and tries to establish whether the change satisfies them. It has access to the diff, the surrounding code, the tests, sometimes the history. It does not have access to a running system, real data, a user, a load profile, or next Tuesday. There is a large class of perfectly reasonable acceptance criteria whose truth simply is not present in the artefact being examined.
For those, there are exactly two possible behaviours. Guess, or decline.
A system that guesses will be right often enough to feel useful and wrong often enough to be dangerous, and crucially you will not be able to tell which is which, because the output looks identical either way. A system that declines is telling you the truth about its own epistemic position, which is the only basis on which you could ever trust the cases where it does answer.
So the decline is not a limitation grudgingly admitted. It is the mechanism that makes the verdicts mean anything.
Which is precisely why it must not be cheap.
Four different things wearing the same label
The reviewer’s four-second shrug happened because “needs human verification” was one bucket. It is really at least four, and they call for entirely different responses.
Out of frame
The claim is about runtime, load, data at scale, or a cross-service flow. No diff could settle it. Route it to a test environment or a person who can observe the system.
Under-specified
The criterion is not checkable by anyone, not just by a machine. "The page feels responsive." The action is to rewrite the criterion, not to verify it.
Evidence not reachable
Checkable in principle, but the relevant code, config or migration is outside the change under review. Widen the frame or link the other change.
Genuinely uncertain
The evidence is present and the reader could not resolve it with confidence. This is the only bucket where "a human should look" is the whole answer.
Look at what happens when you split them. Only the last quadrant is actually a request for human attention on this change. The first is a routing decision. The second is a defect in the specification, and it will recur on every ticket until someone fixes the wording. The third is a scoping problem that may mean the pull request is incomplete.
Three of the four have an action that is not “read the diff more carefully,” and in most teams the second is the largest bucket by a wide margin. If your declines are dominated by under-specification, telling humans to verify more is treating a spec problem with review labour, which is the most expensive medicine available and does not work.
Make the decline produce something
Here is the rule I would put above all others in this area. A decline must leave behind an artefact that did not exist before.
Not a status. An artefact. Something a person can read next week and something that changes the next ticket.
At minimum, a decline should say what claim it could not settle, in the words of the criterion, and what would settle it. “Cannot verify that export completes within thirty seconds for accounts with over a million rows: no performance test exists in this change and the assertion concerns runtime behaviour under data volume.” That sentence is worth more than a hundred generic declines, because it tells you three things: the criterion is legitimate, it is out of frame for static checking, and there is a specific piece of missing infrastructure that would bring it into frame.
The recording step is where the compounding lives. A team that logs its declines with reasons can, after a month, sort them and discover that eleven of their declines are the same three phrases appearing in every ticket written by the same group. That is a fixable problem with a one-hour intervention. Without the log it is invisible, and instead the team concludes that verification “does not really work for our kind of work,” which is the conclusion I hear most often and is almost always wrong in the specific way I have just described.
Who the decline goes to matters more than the decline
A decline routed to “the pull request” is routed to nobody. Everyone on the review will assume that whoever cares most will look, and the person who cares most about merging is the author.
The fix is unglamorous: declines need an addressee, and the addressee should usually not be the author of the change.
For the out-of-frame bucket, the addressee is whoever owns the environment where the claim could be checked. That might mean the criterion becomes a test to be written, or a manual check on staging with a named owner, or an item on a release checklist. What it must not mean is that it evaporates because the pull request merged.
For the under-specified bucket, the addressee is whoever owns the criteria for that area, which in most teams is a product manager or tech lead. This is the loop that improves the backlog, and it is worth being slightly annoying about, because every criterion fixed at the source stops producing declines forever.
For the genuinely uncertain bucket, the addressee is a reviewer with domain context, and this is the only bucket where the traditional review process is the right destination.
The relationship with your confidence threshold
Most verification layers have a confidence dial. Above the line, act. Below it, route to a person. It is tempting to treat “needs human verification” as simply what happens below the line, and to manage the whole problem by moving the number.
That conflates two different things.
Confidence is about how sure the reader is of its own reading. Frame is about whether the answer exists in the material at all. A criterion about response time under load is not a low-confidence case. It is an unanswerable case, and no amount of model improvement or threshold tuning will make it answerable from a diff. If your system expresses the second as the first, then raising the threshold makes it decline more of the answerable cases while still leaving it guessing at the unanswerable ones, which is the worst of both.
One dial for everything
- Everything below the line is "needs human verification"
- Unanswerable claims and uncertain readings look identical
- Tuning the number changes volume, not quality
- Reviewers learn the label carries no information
Frame first, then confidence
- Is the answer present in this material at all
- If not, route by reason and do not score it
- If yes, then apply the confidence threshold
- Declines stay rare enough to be worth reading
The practical consequence: measure the two separately. Out-of-frame rate tells you about your criteria and your test infrastructure. Low-confidence rate tells you about the reader and the codebase. They move for different reasons and they have different fixes.
What “and mean it” actually costs
The title of this piece has a demand in it, so let me be concrete about the price.
Meaning it costs review time you were not spending before. If ten percent of criteria come back needing a human and you are merging a lot of changes, that is real hours, and they land on the people who are already the constraint. Any team that adopts this without budgeting for it will resolve the tension the way that reviewer did, by clicking through, and will then have paid for a gate that gates nothing.
Meaning it also costs a norm that is genuinely uncomfortable: someone has to be allowed to hold a merge. If the culture is that the gate is advisory and shipping is the only thing measured, the decline will lose every time, and it should be obvious to everyone that the gate exists for decoration. Better to be honest about that than to run the ceremony.
The cheapest way to buy both is to be ruthless about volume. Declines should be rare and specific. If more than a small fraction of your criteria come back unanswerable, do not scale up human verification. Go and fix the criteria, because the underlying condition is that your acceptance criteria are describing things nobody can check, and that is a problem with the specification rather than the verification.
Where this breaks down
The honest limits, in rough order of how often they bite.
First, the classification into four buckets is itself a judgement, made by the same reader whose judgement you were hedging against. A system that puts a claim in “under-specified” when it is really “out of frame” sends the work to the wrong person, and now you have added a routing error on top of an uncertainty. In practice the boundary between under-specified and genuinely uncertain is fuzzy, and reasonable readers disagree. I do not have a clean answer to this other than sampling the classifications occasionally, which is more overhead.
Second, insisting that every decline produce an artefact will, in a busy team, produce a backlog of decline records that nobody reads. Logging is not the same as acting. If the log is not reviewed at a stated cadence by a named person, you have built a landfill with good intentions, and the honest version of this article would say that most teams do not sustain the review.
Third, there is a real argument that on high-volume, low-risk changes the right policy is to not check at all rather than to check and decline. A gate that fires constantly on dependency bumps and copy changes trains people to ignore it, and that trained ignorance transfers to the changes that matter. Scoping the gate narrowly is often better than making it more articulate, and this piece has been arguing the articulate route because it is the more interesting one, not always because it is the right one.
Fourth, and worth sitting with: a rigorous decline discipline can make a team slower in a way that is not compensated. If your criteria are decent, your reviewers are experienced, and your change failure rate is already low, the gate may be catching things that your existing process would have caught anyway, one step later, at less cost. The value of formal verification is highest where the volume of change has outrun human attention. Below that point it is ceremony, and the fact that it is well-designed ceremony does not redeem it.
The takeaway
A decline is only worth something if it is specific, classified, addressed to someone, and recorded in a way that improves the next ticket. Split “needs human verification” into out of frame, under-specified, evidence not reachable, and genuinely uncertain, because three of those four are not requests for review at all.
Keep declines rare, treat a rising decline rate as a signal about your criteria rather than your reviewers, and never let the label be a checkbox that a tired person can clear in four seconds.
The next piece takes the obvious next step. If acceptance criteria are the thing being verified, the thing agents build from, and the thing that decides whether a merge happens, then they are not a field on a ticket. They are the product specification, and the last document anyone called a spec has been stale for a year.