Incident response when an agent caused the incident

The blameless postmortem is one of the better ideas our industry has had. It rests on a specific and well-supported claim: people do not cause incidents through carelessness, they act reasonably given the information available, and the interesting question is what made a reasonable action produce a bad outcome. Ask why the system permitted the mistake rather than why the person made it, and you get fixes instead of fear.

The practice has a dependency that is invisible while it holds. It assumes there was a reasoning process to reconstruct. “What did you believe when you ran that command” is the question that unlocks the whole investigation, and it has an answer because a mind was present.

When a change was generated rather than authored, that question has no answer. Not a hidden one, not one that needs coaxing out. There was no belief.

What is actually missing

It is worth being precise, because the loss is narrower than it first appears and the parts that survive are the parts that matter most.

My honest assessment is that this is close to a wash, and possibly an improvement, for a reason that is slightly deflating: human reasoning reconstructed after an incident was never very reliable. People misremember. They rationalise. They produce a coherent account of a decision that was actually made quickly and without much thought, because coherent accounts are what the format rewards. A great deal of postmortem narrative is confabulation performed sincerely.

The criteria record does not have that problem. It says what the change was supposed to do, and it says what was checked. Both are exactly true.

Reframing the central question

The reframe that makes agent-caused incidents tractable is to stop asking why the change was wrong and start asking why nothing caught it.

This is not a rhetorical trick. It reflects where the causal leverage actually is. A generated change that was wrong is an expected event with a known base rate; treating each instance as surprising is a category error, like being surprised by a specific coin flip. What is not expected, and what is worth an investigation, is that a wrong change passed every gate you built.

So the investigation runs backwards through the chain, and at each link asks what should have stopped this.

The distinction between a missing criterion and a false pass deserves emphasis, because teams routinely conflate them and they call for entirely different responses.

A missing criterion means nobody thought to state the property that failed. That is ordinary and expected; the fix is to add it, and to ask whether the same gap exists on similar paths. A false pass means the property was stated, the system said it held, and it did not. That is a verification defect, it undermines every green result you have, and it deserves an urgency that missing criteria do not. In my experience false passes get logged as “improve test coverage” and buried, which is the single worst habit in this whole area.

The generalisation step is the whole value

Here is what separates a review that produces improvement from one that produces a ticket.

A generated defect is almost never a one-off. The same model, given similar tasks in a similar codebase, will produce the same class of error repeatedly. If the incident was caused by a change that assumed a list was non-empty, there are other generated changes in your repository making the same assumption right now, and there will be more next week.

So the correct output of the review is not a fix. It is a rule, applied everywhere, plus a search for existing instances.

Incident:   INC-2291, checkout 500s, 34 minutes
Change:     PR 8814, generated, criteria passed, 1 human approver
Instance fix:      handle empty basket in totals()          <- necessary
Class fix:         criterion “collection ops handle empty”   <- the actual output
applied to all payment-path work
Retro-search:      17 other generated changes with the
same unguarded pattern, 3 reachable
Verification fix:  the passing check never exercised the
empty case; fixture always had 2 items    <- false pass

That retro-search line is the habit worth building. Once you know the shape of a generated defect, finding its siblings is a search problem, and it is one of the few genuinely new capabilities this era offers incident response. Human defects were idiosyncratic and did not generalise. Generated ones cluster.

Accountability, without pretending

There is an uncomfortable question underneath all of this that people avoid in writing and argue about in private: who is responsible?

The evasive answers are that the agent is responsible, which is meaningless, or that the engineer who prompted it is responsible, which is unfair in a way that will destroy the practice’s honesty within two incidents. Blameless review works because people tell the truth, and people tell the truth when truth-telling is safe.

The position I find defensible: accountability sits with the system owners for the design of the gates, and with the organisation for the risk appetite those gates encode. Choosing to let low-risk changes merge on green is a decision with an expected loss attached, taken deliberately, presumably by someone senior. When that expected loss materialises, the correct response is to check whether the risk line was drawn in the right place, not to find the individual nearest the event.

The failure mode is real and predictable: an incident occurs, discomfort needs somewhere to go, and it lands on whoever prompted the change. Do that once and every engineer in the building starts reviewing every generated line defensively, which eliminates the entire benefit and does not improve safety, because defensive review at volume is skimming.

Where this breaks down

Rate matters more than any individual case. Everything above is about learning from one incident. The strategic question is whether the incident rate per change is rising, and that requires measuring changes and incidents together over time. Most organisations track incidents and not the denominator, which makes the trend uninterpretable.

A perfect record can mislead confidently. When the chain shows criteria passed, there is a strong pull toward concluding the change was innocent and searching elsewhere. Criteria passing means the stated conditions held, which is a much weaker claim than correctness. I have watched a team lose forty minutes to this while the record was accurate throughout.

Not every incident has a change to blame. Traffic, dependencies, certificates, capacity. The whole chain-walking approach applies only when something changed, and reaching for it reflexively wastes time during the incidents where nothing did.

Generalisation can be over-applied. Turning every incident into a universal rule produces, within a year, a check suite so restrictive that legitimate changes fail routinely and engineers learn to bypass it. Rules need expiry dates and someone willing to remove them, and almost nobody removes a rule that was created after an outage.

The honest accountability position requires senior cover. Saying “the system owner is accountable for the gates” only works if someone senior actually accepts it when a customer is angry. If that support is absent, the practice collapses into blame regardless of what the documentation says.

The takeaway

Postmortems lose the reconstructed mental state and keep the decision record, which was always the more reliable of the two. That trade is better than it feels.

Ask why nothing caught it rather than why it was wrong. Distinguish a missing criterion from a false pass, and treat the second as the serious finding it is. Produce a rule and a retro-search rather than a fix, because generated defects cluster in a way human ones never did. And put accountability on the design of the gates rather than on whoever was nearest, or people will stop telling you the truth.

If you take one thing into next week: take your most recent incident caused by a change, and ask what criterion, applied to every change on that path, would have caught it. Then find out how many existing changes would fail it.

This closes the governance series. The next set turns to economics, starting with the question every executive asks and almost nobody answers accurately: what this actually costs.