What happens to DORA when you move to the ADLC

An engineering director I know keeps a slide that has not changed in three years. Four numbers, four arrows, all green. Deployment frequency up. Lead time down. Change failure rate flat. Time to restore down. He presents it every quarter and nobody has ever asked a hard question about it, which he used to think was because the numbers were good.

Last quarter the numbers got dramatically better. Deployment frequency roughly doubled. Lead time fell by more than half. He did not do anything to cause it. His team adopted coding agents in earnest around the start of the quarter, and the slide simply improved on its own.

He asked me whether he should be pleased. That is a better question than it sounds, and the honest answer is that the slide had stopped describing his organisation somewhere around week three and he had no way to tell from the slide itself.

This is the first piece in a series about metrics under the Agentic Development Life Cycle. It is worth starting with DORA, because DORA is the measurement framework almost everyone has, and because what happens to it under agentic development is not that it becomes wrong. It becomes ambiguous, which is harder to notice and worse to act on.

What the four keys were actually measuring

The four keys are deployment frequency, lead time for changes, change failure rate, and time to restore service. They came out of a body of research into what distinguishes high-performing software organisations from low-performing ones, and the reason they caught on is that they are unusually hard to game compared to the alternatives that were circulating at the time. Story points can be inflated by fiat. Velocity is a unit of agreement, not a unit of output. Lines of code is a joke. The four keys, by contrast, are anchored to events that either happened or did not: a deploy went out, a change broke something, service was restored.

But hardness-to-game is not the same as meaning-in-all-contexts, and the meaning of the four keys always came from a specific model of how software gets made.

That model went roughly like this. A human decides something needs to change. A human works out how. A human writes it, and the writing is slow and expensive, so the writing is the bottleneck. Everything downstream of the writing is process overhead: review queues, environment contention, release trains, change advisory boards. Under that model, deployment frequency is a proxy for how much of your organisation’s process overhead you have managed to remove, and lead time is a proxy for how long a decision sits in a queue before it becomes real. Both are measuring friction around a fixed-cost creative act.

That is what made them useful. If your deployment frequency was low, it was because something in your system was preventing work that already existed from reaching users. Finding and removing that something was almost always worth doing.

Take away the fixed cost of the creative act and the proxies come loose from what they were proxying.

The specific thing that breaks

Under the ADLC, code production stops being the bottleneck. That statement gets thrown around loosely, so let me be precise about what it does to each key.

Deployment frequency measures throughput of changes, not throughput of decisions. When a human wrote every change, the number of changes flowing through your pipeline was tightly coupled to the number of decisions your organisation had made, because each change cost enough human effort that nobody produced one accidentally. That coupling is now broken. A team can merge a very large number of small changes derived from a small number of decisions. The metric goes up. The organisation’s actual rate of resolving open questions may be flat or falling.

Lead time for changes measures the gap between two events that no longer bracket the work. Classically, lead time ran from commit to production, and the interesting part of that window was queueing: waiting on review, waiting on a release window, waiting on a shared environment. When an agent produces a change in twenty minutes, the commit-to-production window contracts sharply and the metric improves. But the time between someone realising a thing needed doing and that thing being agreed, specified well enough to be built correctly, and accepted as done did not necessarily move at all. The measured window shrank. The real one may not have.

Change failure rate measures a denominator that just exploded. This is the subtle one and I will give it a whole piece later in the series, so I will keep it short here: the rate is failures over changes, and if the change count multiplies while the absolute number of incidents stays roughly constant, the rate falls. It looks like quality improved. Nothing about the quality of your system changed. The arithmetic changed.

Time to restore is the most robust of the four, and even it shifts. Restoring service is still restoring service, and it is still measured in wall clock time against something users noticed. But the diagnostic phase of an incident, working out which change caused this, gets meaningfully harder when the change set for the last twenty-four hours is five times larger and much of it was not held in any human’s head at the moment it was written.

So: two of the four improve mechanically without any underlying improvement, one becomes ambiguous, and one gets harder in a way the metric does not show.

Why this is not a reason to abandon DORA

I want to be careful here, because there is a genre of writing that says “metric X is broken, throw it out,” and it is almost always wrong. The four keys are still the best set of operational health indicators most organisations have. Their failure mode under the ADLC is not that they lie. It is that they become underdetermined: the same reading is now consistent with several very different underlying realities, and the metric no longer tells you which one you are in.

An underdetermined metric is fixable. You fix it by pinning down its basis.

Consider deployment frequency at a team that runs continuous deployment straight off main. Every merge is a deploy, so merges and deploys are the same event and the metric is honest. Now consider a team that batches into a weekly release. Merges are not deploys. If you count merges you will get a number that flatters the team’s actual delivery cadence enormously, and under the ADLC that flattery multiplies, because merge volume is exactly what agentic development inflates.

The metric is the same word in both cases. The meaning is completely different. Most dashboards do not tell you which one you are looking at.

So the requirement to hold any delivery dashboard to, including ours, is that every metric states its own basis on the surface where it is displayed, not in documentation and not in a tooltip. Deployment frequency derived from merges says it is derived from merges. Lead time derived from pull-request-opened to pull-request-merged says that, rather than claiming to be commit-to-production. It is a small thing that changes how the number gets used, because it makes the reader do the translation instead of quietly assuming there is none.

The other half of that decision was more painful to implement and matters more. When there is no data to compute a metric from, the metric returns nothing at all rather than zero, and the surface says so. A team with no reverts in the window does not have a zero percent change failure rate. It has an unknown change failure rate and a small sample. Those are not the same claim, and the gap between them is where most bad quarterly decisions come from.

Deployment frequency   14.2 / day
  basis: merges to main (continuous deployment)
Deployment frequency   14.2 / day
basis: merges to main; deploys are batched weekly
Time to restore        no data
basis: no incident records linked in this window

The reading you actually want

If you keep the four keys and pin down their basis, what should you expect to see when a team moves to the ADLC in earnest? Here is the pattern I would treat as normal, offered as a shape rather than a set of numbers, because I have no defensible numbers to give you and neither does anyone else yet.

Deployment frequency rises, quickly and substantially. Lead time as conventionally measured falls, also quickly. Change failure rate drifts downward for arithmetic reasons while the absolute count of incidents holds steady or ticks up slightly. Time to restore holds roughly flat, with more variance than before, because the easy incidents get diagnosed as fast as ever and the hard ones get harder.

That combination looks, on a slide, like a team that has improved on every axis. Under the surface it is a team that has become much faster at producing changes and has not yet become any better at deciding which changes to produce or at knowing when something has gone wrong.

Which might be fine. It might be exactly what you wanted for a quarter. But you should be choosing that deliberately, and the slide as normally constructed does not give you the chance to.

The question that separates the two readings

There is one diagnostic I have found useful and it does not require new instrumentation. Take your deployment frequency for the quarter and your count of completed, accepted units of user-visible work for the same quarter. Divide.

Under the old model that ratio was reasonably stable, because changes and decisions were coupled. Watch what it does over the quarters in which agent adoption ramps. If deploys per accepted outcome climbs steeply, you are producing more motion per unit of resolved intent. That is not automatically bad. Small changes are good practice and a rising ratio can simply mean better decomposition. But it can also mean churn: rework, half-built features, changes that get made and then unmade, agents circling a poorly specified requirement.

The ratio does not tell you which. It tells you to go and look, which is what a metric is for.

Where this breaks down

I should be honest about the limits of this argument.

The first limit is that everything above assumes a real shift in how your code gets produced. If agents are writing a modest slice of your changes and a human still holds the intent behind most pull requests, the coupling between changes and decisions is stretched rather than severed, and your existing DORA reading is probably still doing its job. Rebuilding your measurement approach in that situation is a cost with no return. Wait until the coupling actually breaks.

The second is that I have described the mechanical inflation of deployment frequency as if it were purely an artefact, and that is unfair. Some of the increase is real. Teams that adopt agents well genuinely do ship more, decompose work into smaller safer pieces, and clear backlogs of small valuable changes that were never worth a human’s afternoon. Treating the entire rise as noise would be as wrong as treating all of it as signal. The uncomfortable truth is that the metric mixes the two and nothing I have described separates them cleanly.

The third limit is more fundamental. The argument that DORA has become underdetermined implies there is a better-determined measure waiting to be adopted. I am not sure there is, not in the sense of a single number you can put on a slide. The things that actually got scarce under the ADLC are decision quality, specification quality and review capacity, and none of those has a clean event stream to count. The candidates I will spend the back half of this series on, rework rate, definition-of-done adherence, drift, are better than nothing and each of them is gameable in ways the four keys are not. Anyone who tells you they have a clean replacement is selling something, and yes, that includes us.

The fourth is a caution about the ratio diagnostic above. It depends on your board being accurate about what was accepted, and the whole premise of the ADLC is that boards drift. Using a drifting artefact as the denominator of your health check is circular. If your board is unreliable, fix that before you trust the ratio, and be aware that fixing it will itself change the number.

The takeaway

DORA does not break under the ADLC. It becomes ambiguous. Deployment frequency and lead time improve for mechanical reasons that have nothing to do with your organisation getting better at delivering. Change failure rate falls because its denominator grew. Time to restore stays flat while the work behind it gets harder.

The fix is not to throw the four keys away. It is to make every metric declare what it is actually computed from, refuse to emit a number when there is no data behind it, and pair the operational four with at least one measure of whether the extra motion is converting into resolved intent.

The next piece takes deployment frequency on its own, and asks what you are really looking at when merge volume outpaces the rate at which your organisation makes decisions.