Coverage was always a weak signal. Now it is a dangerous one.

Line coverage has never measured whether anything was verified. It measures whether a line was executed while the suite ran. A test that calls a function and asserts nothing at all produces the same coverage as a test that checks every branch of its output.

Everybody in the industry has known this for twenty years, and everybody kept using coverage anyway, for a defensible reason: getting to eighty-five percent was expensive. It took hours of someone’s attention. So the number was a poor measure of verification and a decent measure of investment, and investment correlated with quality closely enough to be useful.

My position: that correlation is gone, and coverage has crossed from weak signal to actively harmful one. It now rises fastest exactly where code is least understood, and used as a merge gate it creates a direct mechanical incentive to generate tests that assert whatever the code already does. If you are still reporting a coverage percentage upward, you are reporting a number that has inverted its meaning.

The proxy and the cost that made it work

Every effort-based proxy in software works the same way. The metric does not measure the thing you care about; it measures something that was expensive to fake, and the expense is what carried the information.

Lines of code was a proxy for output. Test count and coverage were proxies for care. Commit frequency was a proxy for engagement. None of them measured the underlying quality directly. All of them worked reasonably well while producing them required human hours.

When the production cost of a proxy collapses, the proxy does not degrade gracefully. It inverts, because the cheapest way to move the number is now available to everyone, including to processes with no understanding behind them at all. Anyone still reading the number as evidence is reading noise with a confident interface.

The gate creates the pathology

The specific harm is not that coverage is uninformative. Uninformative metrics are merely wasteful. The harm is what happens when it is enforced.

A merge gate that requires coverage not to drop is now trivially satisfiable by generating tests against the code you just wrote. The generated tests will pass, because they were derived from the implementation. Coverage goes up. The gate goes green. Nothing has been verified, and the suite has grown by however many files it took to clear the threshold.

test("processes refund", () => {
  const result = processRefund(order, { reason: "damaged" });
  expect(result.status).toBe("pending");
  expect(result.processedAt).toBeNull();
});

There is a cost dimension too, which is the part I care about professionally. Those tests run on every pull request, forever. They add CI minutes, they add wall-clock time to every developer’s feedback loop, and they add maintenance whenever the implementation is refactored, at which point they break and get “fixed” by updating the expected values. You are paying a recurring bill for an artefact that was generated to satisfy a threshold.

What to report instead

The instinct when a metric fails is to find a replacement metric that fits in the same slot on the same dashboard. Resist that, because part of the problem is having a single number in that slot at all.

Mutation score, on a narrow scope. Introduce small changes to the code and see whether the suite notices. This measures verification rather than execution, which is the actual question. It is expensive to run, so run it on the ten percent of the codebase where being wrong is expensive: money, permissions, data destruction. A mutation score on your payments module is worth more than a coverage percentage across everything.

What the suite has actually caught. Over the last quarter, how many defects were found by the test suite before they reached production, and where. This is a lagging measure, it is a bit fiddly to collect, and it is the only number in this list that is about outcomes rather than about the artefact.

Assertion provenance. For your high-consequence paths, what proportion of assertions trace to a stated acceptance criterion rather than to the implementation. This is the one that most directly addresses the failure described above, and almost nobody can measure it today, because there is nowhere in the usual toolchain to record it.

Suite cost. Minutes of CI per pull request, and its trend. Not a quality metric, but it is the one that surfaces uncontrolled growth, and it is the number a platform team can act on immediately.

If you want a single sentence for leadership, it is not a percentage. It is something closer to: on the paths where being wrong is expensive, here is what our tests would catch, and here is what they would not.

The transition problem

Removing a coverage gate is politically harder than it should be, because it reads as lowering the bar. Two things make it easier.

Replace it rather than delete it. A gate that says “changes to these fifteen high-radius paths require a mutation score above X” is a stricter bar than eighty percent coverage across the repository, and it is defensible in a room where someone is worried you are relaxing standards.

And keep coverage as a floor for the paths that have none at all, where zero coverage genuinely tells you something: nobody has been near this with a test. Coverage is a reasonable detector of complete absence. It is a terrible detector of adequacy. Use it for the first and stop using it for the second.

Where this breaks down

Coverage is genuinely useful at the low end and I have skated past that. The difference between zero and forty percent on a module is real information. Somebody has or has not thought about testing this at all. Discarding the metric entirely loses a cheap detector of neglected areas, and the honest position is narrower than “stop measuring coverage”.

Mutation testing is expensive, slow and easy to abandon. It burns compute, produces findings that take judgement to triage, and reliably becomes a report nobody reads by the second quarter. Recommending it is easy. Every team I know that has sustained it has had one person personally invested in keeping it alive, which is not a strategy.

Removing the gate will lower quality in some teams. A weak proxy enforced consistently can outperform a good metric nobody enforces. If your organisation’s real problem is that some teams write no tests at all, a coverage floor is doing useful work despite being philosophically indefensible, and taking it away without replacing it will make things worse.

Assertion provenance sounds tidy and is close to unimplementable today. No mainstream toolchain records where an assertion came from. Getting it means either a convention people maintain by hand, which decays, or tooling that does not really exist yet. I am recommending something that is currently more of a direction than a practice, and I should say so plainly.

And I have a commercial interest in the direction I am pointing. We sell tooling that connects acceptance criteria to what actually changed, so an argument that assertions should trace to criteria happens to describe a product category we are in. Discount it accordingly. The mutation-testing-on-critical-paths recommendation involves no vendor and is where I would start regardless.

The takeaway

Coverage measured execution, and it survived as a proxy for care only because reaching a high number used to cost human hours. That cost is gone, so the number now rises fastest in exactly the code nobody has understood, and enforcing it as a gate rewards generating assertions from the implementation you are trying to check.

Stop reporting a repository-wide percentage. Measure whether your suite would notice if the code were wrong, on the paths where being wrong is expensive. Keep coverage only as a detector of total absence.

If you take one thing into next week: pick your highest-consequence module, change a comparison operator in it deliberately, and run the suite. If it stays green, you have learnt more about your testing than a coverage report has told you all year.