Is Code Coverage a Useless Metric?

Should You Care About Your Test Coverage? Should Anyone?

Photo by Jon Tyson on Unsplash

Many are the smug developers who laud their high coverage over their peers. And many are the cynics who sneer at the mention of any such metric.

I’d like to think that most of us sit somewhere between those two positions, recognizing the value of metrics without becoming a slave to them.

I recently mentioned “100% test coverage” in a sentence. I forget the exact context, but I believe I was pointing out that if you follow TDD to the letter, it should be impossible not to get 100%.

A colleague stood up to peer over his monitor at me, something primal in his brain, angered by the mention of “100% test coverage”. We had a friendly debate on the subject, and I resolved to write down what we discussed.

Metrics

Test coverage is a by-product of good practice, and as such, your development team probably shouldn’t focus on it.

As a developer, you probably know already if you are testing enough, so having a metric that confirms or refutes that belief is unnecessary. If the metric says that you aren’t testing enough, but you believe you are, should you write more tests? Probably not.

Your boss, on the other hand, or their boss, shouldn’t have to get into the weeds to understand if your code is robust.

Like it or not, your boss wants to know that you are doing a good job. This probably includes testing your code, which I say on the assumption that your boss values automated testing.

Let’s not dwell on the possibility that your boss does not value tests, since coverage isn’t going to help in that case.

If anyone (your boss included) wants to know how well you test your code, they have three options:

1. Talk to you about it.
2. Check your code for themselves.
3. Look at some kind of high-level metric that gives them a rough idea.

Option one is viable, but relies on both trust and shared understanding. I’ve worked with some teams who claimed to be writing enough tests, yet every pull request came with implementation but no tests.