Lead time for changes: the metric agents distort most

A platform team I talked to had a lead time of about four hours. Pull request opened to pull request merged, measured honestly, no massaging. Their previous year’s figure had been closer to two days. On any conventional reading they had made an extraordinary improvement in a single year.

I asked how long it took, from someone first saying a thing should exist to that thing being live and accepted. They thought about it and said probably five or six weeks, same as always. Maybe slightly worse, because there was more in flight now and acceptance was backing up.

Nobody in that conversation was confused or being evasive. The four-hour number was correct. The six-week number was also correct. They are measurements of different intervals, and for most of the history of software delivery those intervals were close enough in shape that conflating them did no harm. Agents pulled them apart.

Of the four keys, lead time is the one where this pulling-apart is most severe, and it is the one I would be most careful about presenting to anyone who makes decisions based on it.

What the interval was supposed to contain

Lead time for changes is classically defined as the time from code committed to code running in production. Its purpose was to measure how long finished work waits.

Think about what filled that window under the old model. Review latency, mostly: a pull request sitting for a day because the one person who understands that subsystem is in meetings. Then CI, which used to be genuinely slow. Then whatever release governance you had, which for a lot of organisations was the dominant term. Then environment contention. Then, if you were unlucky, a change advisory board that met on Wednesdays.

Almost all of that is queueing. It is time during which the work is complete and nothing is happening to it. That is what made the metric so actionable: every hour you removed from lead time was an hour of pure waste removed, and the improvements were durable because you got them by building things, not by pushing people.

The critical assumption, though, is the one at the start of the window. Commit. The metric begins when the code exists, which under the old model was a reasonable place to start, because the act of producing the code was the visible, effortful, human part that everyone already understood. Measuring the waiting was the value add.

Where the time went instead

Under the ADLC, the interior of that window collapses. Not because you fixed anything, but because the objects moving through it changed shape.

The changes are smaller, so review is faster. The changes arrive with tests already written, so CI passes more often on the first attempt. The agent that wrote the change is available to revise it immediately rather than in whatever gap its author has between meetings, so review round trips that used to span a day now span twenty minutes. Feature flags decouple merge from release, so the release governance term drops out. Four hours from open to merge is entirely plausible and entirely real.

Meanwhile, look at where time is now actually spent for that platform team. Deciding what to build. Getting the requirement precise enough that the agent builds the right thing. Waiting for someone with the authority to say yes to look at the finished work. Discovering, three weeks later, that the acceptance criteria were ambiguous and the thing built to the letter of them is not the thing anybody wanted.

None of that is inside the measured window. All of it got worse, or at least did not get better, and some of it got worse specifically because throughput upstream increased and acceptance capacity did not.

So lead time as measured improved by a factor of ten while lead time as experienced by whoever asked for the change did not move. That is not a metric being wrong. That is a metric measuring a shrinking fraction of the thing you care about, and reporting the shrinkage as success.

Three different intervals, one word

It is worth naming the intervals separately, because the ambiguity does real damage in conversations between engineering and everybody else.

Cycle time, roughly: pull request opened to merged. This is what most tooling actually computes, including ours, because open and merge are unambiguous events available from any source control provider. It is a good measure of review and integration health. It is not lead time in the DORA sense, and calling it that is where a lot of the trouble starts.

Delivery lead time, the DORA definition: commit to running in production. Longer than cycle time by however much your release process adds. For a continuous deployment team the two nearly coincide. For a batching team they can differ by a week.

Concept-to-cash, or whatever you want to call the honest end-to-end: the moment a need is articulated to the moment it is live and accepted. This is the interval a business stakeholder means when they ask how long something takes. It contains all the specification, prioritisation, queueing-for-attention and acceptance time that the other two exclude.

Under the old model these three were correlated enough that improving one usually meant improving the others. Under the ADLC the first two collapse and the third does not, which means the correlation that justified using the short one as a proxy for the long one has broken. If you report cycle time to a board and they hear concept-to-cash, you have not communicated, you have misled, and you did it with a true number.

This is a large part of why we made our lead time metric say, on its face, that it is derived from pull request opened to pull request merged. It is a weaker claim than “lead time for changes” and it is the claim we can actually support from ingested pull requests. The alternative, printing the stronger label over the weaker computation, is the thing that produced the four-hours-versus-six-weeks conversation in the first place.

The distortion has a second mechanism

There is a subtler way agents move this number, and it took me a while to see it.

Lead time is usually reported as a median or a mean over changes in a window. Both are sensitive to the composition of the change population, not just to how fast each change moves. Under the ADLC, the population changes composition dramatically.

Suppose a team used to produce ten pull requests a week: two large risky ones that took days to review, and eight ordinary ones. Now it produces forty: the same two large risky ones, and thirty-eight small ones, many of them mechanical, dependency bumps, test additions, small refactors an agent produced as a byproduct of the real work.

The two hard changes still take days. Nothing about them improved. But they are now two out of forty rather than two out of ten, so their contribution to the median has vanished entirely. The median lead time is now describing the mechanical changes, which were never the ones you needed to worry about.

This is the same denominator effect that hits change failure rate, and it is why medians are dangerous here. If you are going to report lead time under agentic development, report the distribution, or at least the upper percentiles. The ninetieth percentile of lead time is a far more honest description of your team’s experience than the median, because the ninetieth percentile is where the changes that actually strain your review capacity live. Under the old population those changes dominated the average. Now they are a garnish on it.

Before:  2 hard changes (days)  +  8 ordinary  = 10
         the hard ones move the median
After:   2 hard changes (days)  + 38 ordinary  = 40
the hard ones vanish from the median

What to do instead

I would keep measuring cycle time. It is cheap, it is honest about review and integration health, and if it degrades that is meaningful. But I would put three things beside it.

Report the upper percentiles, not just the middle. If the median is four hours and the ninetieth percentile is nine days, those are two different systems operating in the same repository and only one of them is fine.

Measure the acceptance interval separately. Merged to accepted, where accepted means whoever asked for it has agreed the thing is done. Under the old model this interval was short and boring because merge was near the end of the process. Under the ADLC it is frequently the longest interval in the whole system, and almost nobody measures it because it did not used to be worth measuring. If you instrument one new thing after reading this series, make it this one.

Measure the specification interval too, if you can. Request created to acceptance criteria agreed. This is the front of the pipeline and it is where agentic teams are increasingly bottlenecked, because the criteria now have to be precise enough that a literal-minded builder gets it right. Precision takes longer than vagueness. That cost is real and it belongs in the accounting.

Add those together and you have something close to the honest end-to-end number. It will be much worse than your cycle time. That is the point. You cannot improve an interval you have arranged not to look at.

Where this breaks down

Some honest limits on all of the above.

The first is that I have implied the collapse in cycle time is illusory, and it is not. Faster review round trips are a genuine improvement in the working lives of engineers, and shorter integration windows really do reduce merge conflicts, rebasing cost and the general misery of long-lived branches. If your cycle time fell from two days to four hours, something good happened. My complaint is with the inference from that to “we deliver ten times faster,” not with the underlying improvement.

The second is that the acceptance interval is much harder to measure than I have made it sound. Acceptance is often not an event. It is a person forming an opinion, sometimes in a meeting, sometimes never explicitly at all. Where teams do record it, the record is a ticket transition, which means you are measuring the moment someone dragged a card, and cards get dragged in batches on Friday afternoons. You will get a noisy number with a systematic bias toward whatever ceremony cadence you run. It is still worth having. It is not clean.

The third is that the specification interval, measured naively, punishes exactly the behaviour you want. A team that spends three days getting acceptance criteria genuinely precise will show a worse specification interval than one that writes “make checkout work” in ninety seconds, and the second team’s work will fail acceptance repeatedly and cost more overall. If you put specification interval on a dashboard without pairing it with a quality-of-outcome measure, you will optimise for fast vague requirements, which is the single worst thing you can do under the ADLC. I would think carefully before putting this one in front of anyone who manages by dashboard.

Fourth, the percentile advice assumes you have enough changes for percentiles to be stable. A team merging four pull requests a week does not have a meaningful ninetieth percentile, and computing one produces a number that swings wildly on sample size alone. Small teams should look at the actual slow changes individually. There are few enough of them to read.

The takeaway

Lead time was a measurement of how long finished work waits. Agents collapsed the part of the wait we instrumented and left the part we did not, so the metric improves sharply while the interval anyone outside engineering cares about stays where it was.

Keep cycle time, label it as cycle time, and report its upper percentiles rather than its median. Then measure the two intervals that now dominate the honest end-to-end number: specification, and acceptance. Both will look bad. Both are where your delivery time actually lives.

The next piece is about change failure rate, which I think is the one of the four keys most worth defending, and also the one most easily flattered by a denominator that just multiplied.