Java Code Coverage Tools are distinguished in two main categories:
first tools that add statements to the source code and require to recompile the
source code. And second, tools which instrument the byte code either before
or while running it. The goal is to find out which parts of code are tested by
registering the lines of code executed when doing the test.
JCov JCov is the tool which has been
developed and used with Sun JDK from the very beginning of Java: from the version
1.1. JCov is capable of measuring and reporting Java code coverage. JCov is
distributed under the terms of the GNU Public License, version 2, with the
Classpath Exception. JCov has been open-sourced as a part of OpenJDK
codetools project in 2014. JCov is the only code coverage tool working with a
JDK release in development. = Features =
JCov is capable of reporting next types of code coverage:
Block coverage Line coverage
Branch coverage Method coverage
JCov implements two different ways to save the collected data:
into a file on the filesystem onto a server.
JCov is working by the means of instrumenting Java bytecode. JCov
provides two different approaches to instrument the code:
static instrumentation which is done upfront, changing the tested code
dynamic instrumentation which is done on the fly by the means of Java agent.
JCov has a few more distinctive features which include, but not limited to:
Field coverage Abstract API coverage
Direct/indirect coverage Per-test coverage information
Public API and SPI which makes it possible to implement custom filtering
and/or mining the coverage data. = Tools using JCov =
Oracle JDK JCK
Various Java SE and Java ME TCKs Java FX SDK
Java FX Scene Builder JaCoCo
JaCoCo is an open source toolkit for measuring and reporting Java code
coverage. JaCoCo is distributed under the terms of the Eclipse Public License.
It was developed as a replacement for EMMA under the umbrella of the EclEmma
eclipse project. = Features =
JaCoCo offers line and branch coverage. In contrast to Clover, which requires
instrumenting the source code, and Cobertura, which instruments the
bytecode offline, JaCoCo instruments the bytecode while running the code. To do
this it runs as a Java agent, and can be configured to store the collected data
in a file, or send it via TCP. Files from multiple runs or code parts can be
merged easily. Unlike Cobertura and Emma it fully supports Java 7 and Java 8.
= Tools using or including JaCoCo = SonarQube JaCoCo plugin — one of the
defaults for coverage analyses within the code quality management platform
SonarQube EclEmma Eclipse Code Coverage Plugin,
was formerly EMMA based Jenkins JaCoCo Plugin
Netbeans JaCoCo support IntelliJ IDEA since v11
Gradle JaCoCo Plugin Clover
Clover is a Java Code Coverage Analysis application bought and further developed
by Atlassian. It is a commercial product freely available to open source projects
and non-profit institutions. Clover is using a source code
instrumentation technique, which has its advantages and disadvantages. Some
features include historical reporting, huge control over the coverage gathering
process, command line toolset and API for legacy integration and more.
Clover also allows testing time to be reduced by only running the tests that
cover the application code that was modified since the previous build. This
is called Test Optimization and can lead to huge drops in the amount of time
spent waiting for automated tests to complete.
Clover comes with a number of integrations both developed by Atlassian
and by open source community. In order to run Clover from Apache Ant
you can add the following to Ant build.xml file:
and next run: In order to run Clover from Apache Maven
you can add the following to the ~settings.xml:
and next type: Cobertura
Cobertura is an open source tool for measuring code coverage. It does so by
instrumenting the byte code. EMMA
EMMA is an open source toolkit for measuring and reporting Java code
coverage. EMMA is distributed under the terms of Common Public License v1.0.
EMMA is not currently under active development; the last stable release
took place in mid-2005. As replacement, JaCoCo was developed. EMMA works by
wrapping each line of code and each condition with a flag, which is set when
that line is executed. = Features =
instrument classes for coverage either offline or on the fly.
Supported coverage types: class, method, line, basic block. EMMA can detect when
a single source code line is covered only partially.
Coverage stats are aggregated at method, class, package, and "all classes"
levels. Output report types: plain text, HTML,
XML. All report types support drill-down, to a user-controlled detail
depth. The HTML report supports source code linking.
Output reports can highlight items with coverage levels below user-provided
thresholds. Coverage data obtained in different
instrumentation or test runs can be merged.
it is possible to dump or reset coverage data remotely and without a JVM exit.
does not require access to the source code and degrades gracefully with
decreasing amount of debug information available in the input classes.
can instrument individual .class files or entire .jars. Efficient coverage
subset filtering is possible, too. Makefile and ANT build integration are
supported on equal footing. The runtime overhead of added
instrumentation is small and the bytecode instrumentor itself is very
fast. Memory overhead is a few hundred bytes per Java class.
EMMA is 100% pure Java, has no external library dependencies, and works in any
Java 2 JVM. = EMMA-based tools =
Intellij Idea Plugin — Includes code coverage support from its own and an
EMMA plugin SonarQube EMMA plugin — Enables usage of
EMMA analyses within the code quality management platform SonarQube
Google CodePro AnalytiX Jenkins Emma Plugin
Serenity Serenity is an open source toolkit for
measuring and reporting Java code coverage. As well as coverage, major
code metrics are measured:- cyclometric complexity, stability, abstractness and
distance from main. The report data is persisted to an object database, and
made available via Jenkins/Hudson. The interface replicates the Eclipse IDE
interface visually. Serenity dynamically enhances the byte
code, making a post-compile step unnecessary. Ant and Maven projects are
supported. Configuration is done in xml, an Ant example would be:
And a Maven configuration example would be:
For a full example of a configuration please refer to the Jenkins wiki at
i.jenkins-ci.orgJENKINS/Serenity+Plugin. Jenkins slaves as well as Maven multi
module projects are supported. References
^ JaCoCo Mission ^ Patroklos Papapetrou. "Code Coverage
Tools Comparison in Sonar". Only Software matters. Retrieved 3 March
2013. ^ Measure Coverage by Integration Tests
with Sonar – Updated ^ a b "Code Coverage". IntelliJ IDEA
12.0 Web Help. JetBrains. Retrieved 3 March 2013.
^ EclEmma, Eclipse code coverage plugin ^ Jenkins JaCoCo Plugin
^ NetBeans JaCoCo support ^ Gradle JaCoCo Plugin
^ Why does Clover use source code instrumentation?
^ Clover-for-Ant two line integration ^ Clover-for-Maven quick start guide
^ EMMA code coverage files on SourceForge.net
^ Expert Spring MVC and Web Flow; By Seth Ladd, Darren Davison, Steven
Devijver, Colin Yates, p. 289 External links
Clover vs Cobertura vs JaCoCo vs CodeCover vs PITest comparison
JaCoCo vs Clover2 Pick your code coverage tool in Sonar
2.2, a little outdated, as JaCoCo meanwhile supports branch coverage.
SourceForge project page for EMMA