Instrumentation-free measurement: reading the repo instead
Most metrics programmes fail because they ask people to produce new data. The repository already contains almost everything, and it has the one property self-reported data never has.
Every failed metrics initiative I have watched had the same first step. Somebody decides what should be measured, discovers the data does not exist, and asks people to start producing it. Add a label to your pull requests. Fill in the actual-hours field. Tag your commits with the work type. Move the card when you start, not when you finish.
It works for about five weeks. Compliance is high at first because the request is fresh and somebody senior asked. Then it decays, unevenly, and the unevenness is the killer: some teams keep doing it and some do not, so the data is not just incomplete, it is biased toward the teams that comply, which are systematically the teams with slack. By month four the dashboard is built on a sample that correlates with everything except the thing being measured, and quietly nobody looks at it any more.
Compliance is high at first because the request is fresh and somebody senior asked. Then it decays unevenly, and the unevenness is the killer: some teams keep it up and some do not.
The usual lesson drawn is about change management. The right one is that any measurement requiring ongoing human effort measures compliance as much as it measures the thing.
The lesson people usually draw is about change management: better communication, more executive sponsorship, tie it to performance. I think that is the wrong lesson. The right one is that any measurement requiring ongoing human effort will decay in proportion to how busy people are, which means it fails precisely when conditions get interesting.
So build measurement that asks for nothing.
What is already there
The good news is that a software team leaves an extraordinarily detailed trace of itself as a side effect of doing the work, and it does so in a system that cannot be neglected, because neglecting it means not shipping.
A pull request alone carries: who opened it, when, against which branch, which files it touched, how many lines in each direction, when review was requested, who reviewed, what they said, how many rounds, when it merged, whether it was reverted afterwards, and whatever a branch name or a description says about which work item it belongs to. Commits underneath it carry timing and authorship and message. Across a repository over time you get file-level churn, ownership concentration, and which regions of the codebase are hot.
That is enough to compute a great deal without asking anyone for anything.
Deployment frequency, using merges to the default branch as a proxy for deploys. Exact for continuous deployment teams, an overstatement for batching teams, and either way honest as long as the basis is stated.
Lead time, as pull request opened to merged. Cycle time strictly speaking, and it should be labelled as what it is rather than as the DORA definition it approximates.
Change failure rate, as reverts over merges. Narrower than incidents-over-deploys, available everywhere, useful as an early indicator.
Time to restore, as the interval from a high-severity finding being raised to being resolved. Not the same population as user-visible outages, and worth having anyway.
Reconciliation gap and drift, from joining branch and pull request references to work items.
Rework signals, from the pattern of successive modifications to recently merged code.
Review load, from changes per reviewer per period.
Every one of those is derived from artefacts that exist because the team is doing its job. Not one requires a behaviour change, a label, a field, or a reminder in a retrospective.
The property that matters
The argument for repository-derived measurement is usually framed as convenience: no rollout, no compliance problem, works from day one. That is true and it is not the important part.
The important part is that source control is the only artefact in the system that is definitionally true. It is not a description of the work. It is the work. A ticket is a claim about reality. A time entry is a claim about reality. A status field is a claim about reality, made by a person, at a moment, with incentives. The merge commit is reality itself.
Everything else in your measurement stack is a copy, and copies drift. Under the ADLC, copies drift fast, which is the subject of the previous piece. Measurement built on ground truth does not have that failure mode, and that immunity is worth more than the convenience.
There is a second property that follows from it: repository-derived measurement degrades gracefully rather than silently. If your ingestion breaks, you see a gap, because merges stop appearing. If self-reported data breaks, you see plausible-looking numbers computed from whoever still bothers. Visible failure is a feature.
Measurement people maintain
- A claim about reality
- Accurate only when someone updated it
- Drifts, and drifts fast under the ADLC
- Fails silently: the numbers keep arriving
Measurement derived from the repository
- Not a description of the work, the work
- Cannot be neglected without the work stopping
- Does not drift, because nothing is copied
- Fails visibly: merges stop appearing
What it cannot see, and being honest about it
Now the harder half, because a piece arguing that you can read everything from the repository would be a piece I distrust.
Anything that happens before a branch exists. Specification, prioritisation, discussion, the two weeks a request spent in somebody’s inbox. The repository sees work from the moment code starts and is blind to everything upstream, which is exactly where the ADLC moved the constraint. This is the single largest limitation and no amount of clever inference fixes it.
Work that leaves no source control footprint. Infrastructure changed through a console, data pipelines configured in a vendor UI, low-code platforms, anything configured by hand. For teams where a substantial share of real change happens this way, repository-derived metrics under-report severely and in a direction that looks like the team is doing nothing.
Intent. The repository shows that a file changed. It does not show why, unless somebody wrote it down. Nearly every interesting judgement, was this rework or iteration, was this correction or extension, requires intent, and the repository contains only its shadow.
Effort. Lines changed is not effort and never was. Under the ADLC it is not even close, since an agent produces four hundred lines in the time a human produces four. Any metric that treats change size as a proxy for work is now actively misleading, which retires a whole family of measures people used to lean on.
Whether the thing was any good. Nothing in the repository tells you whether users wanted it. That is not a limitation of this approach specifically, but it is worth stating so that repository metrics are never mistaken for outcome metrics.
The honest position is that repository-derived measurement covers the flow plane comprehensively and the intent plane partially, through the links between changes and work items. It has nothing to say about value.
The design rule that follows
If you are going to build measurement this way, one rule matters more than any other: every metric must state what it is computed from, and must return nothing rather than a number when the data is absent.
The first half is about avoiding the mismatch between a strong label and a weak computation. “Lead time for changes” over a computation of pull-request-opened-to-merged invites the reader to believe something the data does not support. “Lead time, derived from PR opened to merged” invites them to apply their own discount, which they are well placed to do because they know their release process and the tool does not.
The second half is less obvious and matters more. A team with no reverts in a window does not have a zero percent change failure rate. It has an unmeasured change failure rate and a small sample. A team with no high-severity findings does not restore service instantly. Emitting zero in these cases manufactures good news out of missing data, and manufactured good news is worse than no news, because it terminates the enquiry.
Returning null is unpopular. Dashboards look unfinished with blanks on them and somebody always asks for the blanks to be filled. Filling them is how you end up with a metrics system whose most confident readings come from its emptiest data. We built GroundTruth’s DORA surface to return null and say why, and it does make the screen look less impressive. It also means every number on it is a number you can act on.
Where this breaks down
Some real limits on the whole approach.
The blind spot is exactly where the ADLC constraint lives. I said it above and it deserves repeating as a limitation rather than a caveat. Repository-derived measurement is best at exactly the things that got cheap and abundant, and worst at the things that got scarce. There is a real risk of building an elegant instrumentation-free system that measures the flow plane beautifully and reinforces the blind spot described earlier in this series. The mitigation is to accept that some things must be captured deliberately, at acceptance time, by a person, and to make that capture as small as humanly possible. One field. One timestamp. Not a form.
Ground truth is only true about itself. A merge is definitionally true as a fact about the repository. Every inference from it, that a merge is a deploy, that a revert is a failure, that a modification to recent code is rework, is a model, and models are wrong in ways that vary by team. The immunity to drift applies to the raw events, not to the interpretations layered on them, and the interpretations are where all the meaning is.
Link quality caps everything on the intent plane. All the reconciliation and drift measurement depends on matching changes to work items, which depends on branch naming or explicit references. Teams with inconsistent conventions get partial coverage, and partial coverage of a link is worse than none, because the unmatched share is not random. It is concentrated in whoever does not follow the convention, which is usually whoever is busiest.
Instrumentation-free is not effort-free. Ingesting a large repository history, keeping it current, handling monorepos, forks and unusual workflows, and resolving identities across systems is substantial work. It just happens to be work done once by a system rather than continuously by every engineer. That is a much better trade, and it is not the same as free.
It can feel like surveillance. Data derived automatically from work people did is data they did not choose to provide. Repository metrics used at the team level to understand flow are one thing. The same data pointed at individuals is another, and the second use will poison the first permanently the moment it happens once. If you cannot commit to team-level use, do not build it.
The takeaway
Measurement that requires ongoing human effort decays fastest when things get busy, which is when you need it. The repository already contains most of what you want, produced as a side effect of doing the work, in the only artefact in your stack that is the work rather than a description of it.
Read that instead. Then follow the one rule that makes it trustworthy: state the basis of every metric on its face, and return nothing rather than a manufactured zero when the data is not there. And stay clear-eyed that this approach is strongest exactly where agentic development made things abundant, and weakest where it made things scarce.
The last piece in this series is about the other end of the problem: how to put any of this in front of a board without either overclaiming or losing the room.