Despite all the advice and the conventions surrounding unit tests,
it can still be hard to know if you wrote them correctly.
The perfect unit test suite would find all bugs in
your code but what finds bugs in your unit tests?
If we could write perfect unit tests,
we probably would just write perfect code in
the first place and not need unit tests at all.
But of course, that is not the case.
How do we evaluate our unit tests for fitness?
One way to test the tests,
so to speak, is to see if they actually run all your code.
This measurement is referred to as code coverage,
or sometimes test coverage.
Coverage is a measurement of what percentage of
your source code is executed by unit tests.
Usually we'll look at lines,
but it's also useful to look at coverage of methods and classes as well.
Let's look at an example of evaluating a unit test for code coverage.
Here is a simple service implementing the classic toy problem, FizzBuzz.
This method takes a single integer and returns the word Fizz,
if the integer is a multiple of three,
Buzz if the integer is a multiple of five,
and FizzBuzz if it's a multiple of both three and five.
This is a silly problem,
but it's good for demonstrating code coverage.
Here's my FizzBuzz service test class and you can see that I've
broken out my test logic into three different methods that each test,
one of the three scenarios I want to cover.
Multiples of three, return Fizz.
Multiples of five, return Buzz,
and multiples of three and five return FizzBuzz.
Let's run the first unit test for code coverage.
In IntelliJ, if I right-click on the method,
I can select the option run with coverage.
A new window appears showing me the class,
method and line percentage coverage organized by package.
If I double-click on this package,
I can see that the FizzBuzz service has 100 percent of its method is tested,
but only 50 percent of the lines.
When I open the FizzBuzz service class again,
notice the red and green bars along the side of the source file.
Green represents lines that had been executed by
my test and red indicates lines that are not executed by my test.
It makes sense that my test for multiples of three executed lines 12,
14, and 15, but not the rest of the method.
Let's go back and run the whole test for coverage now.
We can see that all of my tests pass and now we're up to 87 percent line coverage,
but not quite 100 percent.
Let's see what's missing. Everything in FizzBuzz is covered except for line
19 that's because none of our test
checked numbers that are not a multiple of three or five.
That requirement wasn't mentioned in the comments and so we missed it.
This is fairly common and it's exactly why
unit tests can help clarify requirements for future developers.
Why don't we add a method to complete our test coverage?
We'll make a parameterized test that passes in
some integers that are not multiples of three or five,
and we will assert that the return value
is equal to the integer value of the number we pass in.
Now, when we rerun our test for coverage,
we'll be able to see that we've
achieved 100 percent line coverage of our FizzBuzz service.
An inevitable question with code coverage is what percent coverage should be aimed for?
There are a lot of different opinions on
this subject but I'm going to take the position that
code coverage is not a goal itself and
so there's no particular number you should strive for.
Code coverage is a tool that helps you evaluate if your test miss anything but it's not
a substitute for making sure that
all your requirements and boundary conditions are tested.
Consider our tests for the FizzBuzz service.
Even with 100 percent coverage we tested no unusual inputs.
We didn't test negative numbers,
we didn't test zero or very large numbers.
It's not a complex method and so the testing we did gives us a decent idea
how it works but 100 percent coverage doesn't mean we didn't miss anything.
There are also some methods that do not require code coverage,
like getters and setters,
or perhaps a default constructor.
Insisting on 100 percent coverage or any arbitrary amount of coverage risks
the creation of unnecessary tests that add noise and give a false sense of security.
Use code coverage to help identify any holes
in what you planned to test and focus on writing
good unit tests that meet
our other recommendations and cover all the requirements of your program.