Are you interested in automated testing? If so, become a member of the Java Testing Society!

Creating Code Coverage Reports for Unit And Integration Tests with The JaCoCo Maven Plugin

X-ray image of a hand showing thumbs up

When I started using Java 7, I noticed right away that the Cobertura Maven plugin doesn’t support it. This was a huge problem to me because I used code coverage reports every day.

I did some research and found the JaCoCo code coverage library. It looked interesting and I decided to give it a shot.

The problem was that configuring it was really hard and took a lot of time. I read numerous tutorials just to find out that the instructions given in them did not work for me. Then I ran into this blog post and everything fall into place.

Although that blog post was extremely valuable to me, it is a bit vague. I felt that a more detailed explanation about the usage of the JaCoCo Maven plugin would be valuable.

This blog post describes how we can create code coverage reports for unit and integration tests by using the JaCoCo Maven plugin.

The requirements of our build are following:

  • Our build must create code coverage reports for both unit and integration tests when the tests are run.
  • The code coverage reports must be created in separate directories. In other words, the code coverage report for unit tests must be created into a different directory than the code coverage report for integration tests.

Let’s get started.

Note: The example application of this blog post is based on the example application of my blog post called Integration Testing with Maven. If you have not read it yet, I recommend that you read it before reading this blog post.

Configuring The JaCoCo Maven Plugin

We use the JaCoCo Maven plugin for two purposes:

  1. It provides us an access to the JaCoCo runtime agent which records execution coverage data.
  2. It creates code coverage reports from the execution data recorded by the JaCoCo runtime agent.

We can configure the JaCoCo Maven plugin by following these steps:

  1. Add the JaCoCo Maven plugin to the plugins section of our POM file.
  2. Configure the code coverage report for unit tests.
  3. Configure the code coverage report for integration tests.

These steps are described with more details in the following.

Adding The JaCoCo Maven Plugin to The POM File

We can add the JaCoCo Maven plugin to our POM file by adding the following plugin declaration to its plugins section:

<plugin>
	<groupId>org.jacoco</groupId>
	<artifactId>jacoco-maven-plugin</artifactId>
	<version>0.6.3.201306030806</version>
</plugin>

Let’s move on and find out how we can configure the code coverage report for our unit tests.

Configuring The Code Coverage Report for Unit Tests

We can configure the code coverage report for unit tests by adding two executions to the plugin declaration. These executions are described in the following:

  1. The first execution creates a property which points to the JaCoCo runtime agent. Ensure that the execution data is written to the file target/coverage-reports/jacoco-ut.exec. Set the name of the property to surefireArgLine. The value of this property is passed as a VM argument when our unit tests are run.
  2. The second execution creates the code coverage report for unit tests after unit tests have been run. Ensure that the execution data is read from the file target/coverage-reports/jacoco-ut.exec and that the code coverage report is written to the directory target/site/jacoco-ut.

The relevant part of our plugin configuration looks as follows:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.6.3.201306030806</version>
    <executions>
        <!--
            Prepares the property pointing to the JaCoCo runtime agent which
            is passed as VM argument when Maven the Surefire plugin is executed.
        -->
        <execution>
            <id>pre-unit-test</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
            <configuration>
                <!-- Sets the path to the file which contains the execution data. -->
                <destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
                <!--
                    Sets the name of the property containing the settings
                    for JaCoCo runtime agent.
                -->
                <propertyName>surefireArgLine</propertyName>
            </configuration>
        </execution>
        <!--
            Ensures that the code coverage report for unit tests is created after
            unit tests have been run.
        -->
        <execution>
            <id>post-unit-test</id>
            <phase>test</phase>
            <goals>
                <goal>report</goal>
            </goals>
            <configuration>
                <!-- Sets the path to the file which contains the execution data. -->
                <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
                <!-- Sets the output directory for the code coverage report. -->
                <outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

Let’s find out how we can configure the code coverage report for our integration tests.

Configuring The Code Coverage Report for Integration Tests

We can configure the code coverage report for integration tests by adding two executions to the plugin declaration. These executions are described in the following:

  1. This first execution creates a property which points to the JaCoCo runtime agent. Ensure that the execution data is written to the file target/coverage-reports/jacoco-it.exec. Set the name of the property to failsafeArgLine. The value of this property is passed as a VM argument when our integration tests are run.
  2. Create an execution which creates the code coverage report for integration tests after integration tests have been run. Ensure that the execution data is read from the file target/coverage-reports/jacoco-it.exec and that the code coverage report is written to the directory target/site/jacoco-it.

The relevant part of our plugin configuration looks as follows:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.6.3.201306030806</version>
    <executions>
		<!-- The Executions required by unit tests are omitted. -->
        <!--
            Prepares the property pointing to the JaCoCo runtime agent which
            is passed as VM argument when Maven the Failsafe plugin is executed.
        -->
        <execution>
            <id>pre-integration-test</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
            <configuration>
                <!-- Sets the path to the file which contains the execution data. -->
                <destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
                <!--
                    Sets the name of the property containing the settings
                    for JaCoCo runtime agent.
                -->
                <propertyName>failsafeArgLine</propertyName>
            </configuration>
        </execution>
        <!--
            Ensures that the code coverage report for integration tests after
            integration tests have been run.
        -->
        <execution>
            <id>post-integration-test</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>report</goal>
            </goals>
            <configuration>
                <!-- Sets the path to the file which contains the execution data. -->
                <dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
                <!-- Sets the output directory for the code coverage report. -->
                <outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

That’s it. We have now configured the JaCoCo Maven plugin. Our next step is to configure the Maven Surefire plugin. Let’s find out how we can do this.

Configuring The Maven Surefire Plugin

We use the Maven Surefire plugin to run the unit tests of our example application. Because we want to create a code coverage report for our unit tests, we have to ensure that the JaCoCo agent is running when our unit tests are run. We can ensure this by adding the value of the surefireArgLine property as the value of the argLine configuration parameter.

The configuration of the Maven Surefire plugin looks as follows (the required change is highlighted):

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-surefire-plugin</artifactId>
	<version>2.15</version>
	<configuration>
		<!-- Sets the VM argument line used when unit tests are run. -->
		<argLine>${surefireArgLine}</argLine>
		<!-- Skips unit tests if the value of skip.unit.tests property is true -->
		<skipTests>${skip.unit.tests}</skipTests>
		<!-- Excludes integration tests when unit tests are run. -->
		<excludes>
			<exclude>**/IT*.java</exclude>
		</excludes>
	</configuration>
</plugin>

We are almost done. The only thing left for us to do is to configure the Maven Failsafe plugin. Let’s find out how we can do it.

Configuring The Maven Failsafe Plugin

The integration tests of our example application are run by the Maven Failsafe plugin. Because we want to create a code coverage report for our integration tests, we have to ensure that the JaCoCo agent is running when our integration tests are run. We can do this by adding the value of the failsafeArgLine property as the value of the argLine configuration parameter.

The configuration of the Maven Failsafe plugin looks as follows (the required change is highlighted):

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-failsafe-plugin</artifactId>
	<version>2.15</version>
	<executions>
		<!--
			Ensures that both integration-test and verify goals of the Failsafe Maven
			plugin are executed.
		-->
		<execution>
			<id>integration-tests</id>
			<goals>
				<goal>integration-test</goal>
				<goal>verify</goal>
			</goals>
			<configuration>
				<!-- Sets the VM argument line used when integration tests are run. -->
				<argLine>${failsafeArgLine}</argLine>
				<!--
					Skips integration tests if the value of skip.integration.tests property
					is true
				-->
				<skipTests>${skip.integration.tests}</skipTests>
			</configuration>
		</execution>
	</executions>
</plugin>

Creating Code Coverage Reports

We have now successfully finished the required configuration. Let’s see how we can create code coverage reports for unit and integration tests.

The example application of this blog post has three build profiles which are described in the following:

  • The dev profile is used during development and it is the default profile of our build. When this profile is active, only unit tests are run.
  • The integration-test profile is used to run integration tests.
  • The all-tests profile is used to run both unit and integration tests.

We can create different code coverage reports by running the following commands at command prompt:

  • The command mvn clean test runs unit tests and creates the code coverage report for unit tests to the directory target/site/jacoco-ut.
  • The command mvn clean verify -P integration-test runs integration tests and creates the code coverage report for integration tests to the directory target/site/jacoco-it.
  • The command mvn clean verify -P all-tests runs unit and integration tests and creates code coverage reports for unit and integration tests.

That’s all for today. As always, the example application of this blog post is available at Github.

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He is specialized in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

65 comments… add one

  • Nice writeup!

    Question: How do you fail the build if IT code coverage falls below a certain % ?

    Reply
    • Thanks! You can use the check goal of the JaCoCo Maven plugin to fail the build if the code coverage is lower than a threshold value.

      I haven’t personally used it but I assume that you have to add a new execution which is run after the integration tests have been run (post-integration-test phase). I can try this out tomorrow (is quite late here and I have to sleep a bit before work).

      Reply
  • Nice article that helped me a lot to understand the relations between the invilved tools. But I still have problems to create a process for a multi module project, where all tests are separated from the implementation projects.
    i.e. we have the following project structure:

    RootProject
    pom.xml

    ProjectA
    –> src
    pom.xml

    ProjectATest (depends on ProjectA)
    –> test
    pom.xml

    ProjectB (depends on ProjectA)
    –> src
    pom.xml

    ProjectBTest (depends on ProjectB)
    –> test
    pom.xml

    ProjectITTests (depends on ProjectA and ProjectB)
    –> test
    pom.xml

    Now I want to measure the code coverage with Jacoco. Everything I already tried didn’t work. The code coverage report in the test projects doesn’t contain the coverage of the sources that lie in different projects. i.e. the report of ProjectATest contains no coverage for sources of ProjectA.

    Do you have any idea how to achive code coverage analysis with Jacoco and Maven?

    Reply
    • I have to confess that I haven’t used the JaCoCo Maven plugin in a multi module project like this. However, I did found a few websites which might be helpful to you:

      Reply
    • This is because your sources/classes are not available when JaCoCo is executed. I suppose that the JaCoCo plugin is executed on ProjectATest, ProjectBTest, and ProjectITTests ? Not on ProjectA, and ProjectB (which contains the sources/classes you want to analyse…) ?
      Which kind of tests are performed on these projects ? UT or IT or both ?

      Reply
  • Really interesting article.
    I have to use Jacoco for integration tests that call a Jetty server. I would like Jacoco returns code coverage of the classes used by Jetty. Here is my configuration : https://groups.google.com/forum/#!topic/jacoco/K-PCEg5LLd0
    Have you any experience / idea on how to configure the run of Jetty to make Jacoco returns the server side code coverage ?
    Thanks.

    Reply
    • I have no personal experience from this. However, I have one idea:

      Have you tried to use the run-forked goal of the Jetty Maven plugin?

      This starts Jetty in a new VM, and you can you can pass arguments to the forked VM by using the <jvmArgs> configuration element. You could try to pass the the value of the failSafeArgLine property to the forked VM by adding this line to the configuration section of the Jetty Maven plugin:

      <jvmArgs>${failsafeArgLine}</jvmArgs>

      Let me know if this solved your problem.

      Reply
  • Excellent! Thanks very much Petri, had spent about a day messing around with our maven configuration to get the coverage reports split by Unit and Integration and also had problems including Groovy test results.
    But all good with the help of your article.
    @Stefan, I have a vanilla multi-module project that this is working with so will publish to Github if you still have an issue?
    Thanks again @Petri, best regards, Andrew

    Reply
    • Hi Andrew,

      You are welcome! I am happy to hear that this blog post was useful to you.

      Could you add the url of your vanilla multi-module project after you have published it? I would like to add the url to this blog post (and give kudos to you).

      Reply
  • Hi petri,

    I have been working on this for a week now and I am not sure why I am not getting any code coverage for acceptance test.

    First: We have ant build project.
    Second: We run our acceptance test using selenium

    Steps I have tried to get code coverage:
    Install jacocoant.jar and jacocoagent.jar in lib folder of our project.
    In start-app.sh/start-reg.sh I have mentioned javaagent=../pathto/jacocoagent.jar=destfile=../pathtostore/jacoco.exec,includes=com.abc.*

    So, when I start my servers, it creates jacoco.exec . I execute my acceptance test, I thought after I execute my acceptanceTest, jacocoagent is suppose to generate code coverage. But, my jacoco.exec file is the same. I dont see any increasement on the file size even after the acceptanceTest is completed. I tried shutting down the servers after test is completed but still no change.

    So, I am wondering how can I get code coverage for acceptanceTest. Did I miss anything?

    Reply
    • I haven’t used JaCoCo without Maven but I assume that the process has the same steps.

      First, you have to generate the code coverage data by using the Java Agent.

      When I run the the command mvn clean test (in the root directory of the example application) at command prompt, I see the following log line:

      [INFO] surefireArgLine set to -javaagent:/Users/loke/.m2/repository/org/jacoco/org.jacoco.agent/0.6.3.201306030806/org.jacoco.agent-0.6.3.201306030806-runtime.jar=destfile=/Users/loke/Projects/Java/Blog/maven-examples/code-coverage-jacoco/target/coverage-reports/jacoco-ut.exec

      Is the jacocoagent.jar you mentioned the same jar which is used by my example application? Also, do you pass these arguments to the VM which is used to run your server?

      Second, you have to create the actual code coverage report from the gathered code coverage data. You can create this reports by using Eclemma, Ant or Maven. It seems that there is no command line tool yet.

      I hope that I could help you out although I couldn’t provide the definitive answer.

      Reply
  • Hi Petri,

    are you aware of any way to measure the total coverage of both unit and integration tests? Some of my classes are really easy to unit test so I have unit tests for them, and some are quite hard so I’ve written integration tests. However with the setup you have provided it fails saying that class A has not enough unit test coverage (which is true cause it has integration tests and It’s not that easy to write unit tests) or it fails saying class B has not enough integration test coverage (which is true – it has unit tests!!)..

    Thanks for the nice article and keep up the good work!

    Reply
    • Hi Petar,

      You could try following these steps:

      1. Put your integration tests to the src/main/test directory.
      2. Remove the exclusion from the configuration of Maven Surefire plugin.
      3. Remove the Maven Failsafe plugin from your POM file.

      This should do the trick as long as your integration tests don’t require that the application is deployed to a servlet container or an application server which is run in a separate VM.

      Reply
  • Petri, nice and informative post how to use JaCoCo. If you are working with open source software, you should also check Coveralls, a code coverage hosting service.

    I have created a maven plugin for the Coveralls service that can use either JaCoCo or Cobertura (and Saga soon) as the code coverage tool. Check the GitHub page of the Coveralls maven plugin and Coveralls docs for Java projects for more information.

    Reply
    • Tapio, thanks!

      Coveralls looks interesting, and I will take a closer look at it later today. The maven plugin looks good as well.

      By the way, I hadn’t heard about Saga before. Thank you for mentioning it. Maybe 2014 is the year when I start to write tests for my Javascript code.

      Reply
  • Wow! Thank you very much for the really good description you are giving us.

    As I’m a totally beginner, I am trying to setup Maven/Jacoco in order to publish the code coverage results to Sonar.
    I have 1 question:
    1) How do we set the JaCoCo execution data file? What is that?

    Though, even following your post, I’m dealing with the same problem: maven-compiler doesn’t find the ‘sources’ and I get an error: “No sources to compile” despite that maven-resources-plugin copies the right resources.

    I get the following command line output:

    $ mvn clean test
    [INFO] Scanning for projects…
    [INFO]
    [INFO] ————————————————————————
    [INFO] Building Code Coverage – Maven and JaCoCo running tests 1.0-SNAPSHOT
    [INFO] ————————————————————————
    [INFO]
    [INFO] — maven-clean-plugin:2.5:clean (default-clean) @ example-java-maven —
    [INFO] Deleting sonar_local/src/target
    [INFO]
    [INFO] — jacoco-maven-plugin:0.6.4.201312101107:prepare-agent (pre-unit-test) @ example-java-maven —
    [INFO] surefireArgLine set to -javaagent: .m2/repository/org/jacoco/org.jacoco.agent/0.6.4.201312101107/org.jacoco.agent-0.6.4.201312101107-runtime.jar=destfile=sonar_local/src/target/coverage-reports/jacoco-ut.exec
    [INFO]
    [INFO] — maven-resources-plugin:2.3:resources (default-resources) @ example-java-maven —
    [INFO] Using ‘UTF-8′ encoding to copy filtered resources.
    [INFO] Copying 53 resources
    [INFO] Copying 41 resources
    [INFO]
    [INFO] — maven-compiler-plugin:3.1:compile (default-compile) @ example-java-maven —
    [INFO] No sources to compile
    [INFO]
    [INFO] — build-helper-maven-plugin:1.8:add-test-source (add-integration-test-sources) @ example-java-maven —
    [INFO] Test Source directory:/sonar_local/src/test/tcl added.
    [INFO]
    [INFO] — maven-resources-plugin:2.3:testResources (default-testResources) @ example-java-maven —
    [INFO] Using ‘UTF-8′ encoding to copy filtered resources.
    [INFO] Copying 5 resources
    [INFO]
    [INFO] — maven-compiler-plugin:3.1:testCompile (default-testCompile) @ example-java-maven —
    [INFO] Nothing to compile – all classes are up to date
    [INFO]
    [INFO] — maven-surefire-plugin:2.16:test (default-test) @ example-java-maven —
    [INFO]
    [INFO] — jacoco-maven-plugin:0.6.4.201312101107:report (post-unit-test) @ example-java-maven —
    [INFO] Skipping JaCoCo execution due to missing execution data file
    [INFO] ————————————————————————
    [INFO] BUILD SUCCESS
    [INFO] ————————————————————————
    [INFO] Total time: 3.451s
    [INFO] Finished at: Fri Jan 03 11:48:41 GMT 2014
    [INFO] Final Memory: 12M/110M
    [INFO] ————————————————————————

    I would be really grateful if you could help me.
    Thank you very much

    hAppY NeW YeaR! !

    Reply
    • Hi,

      Thank you for your kind words! I really appreciate them.

      The execution data file contains the code coverage information which is used to create the actual code coverage report. Its location is found from the configuration of the JaCoCo Maven plugin (unit tests and integration tests).

      I have a few questions about your problem:

      • Do you have any source code files in the src/main/java directory?
      • What packaging type are you using in the pom.xml file (If you are using pom packaging type, read this StackOverflow question)?
      Reply
  • Hi,

    Thanks for the helpful post! One question: using this configuration (generating both unit and integration test Jacoco coverage reports) have you tried using it with the maven-site-plugin? I was only able to get either the unit OR the integration reports to display in the generated Maven site.

    Reply
    • Hi,

      I tried to use the maven-site-plugin when I wrote this blog post but I couldn’t get it to generate the both reports. Then I found this pull request. It should fix this problem but at the time it was not merged yet.

      It seems to be merged now. Maybe you could it out and see if it works? Here is a link to the commit which contains an example pom.xml file.

      Reply
      • thanks for the reply Petri. i’ll check it out

        Reply
        • You are welcome!

          I hope that it solves your problem. I should probably try it too because it would be a nice addition to the example application of this blog post.

          Reply
  • Hi Petri,

    I would like to generate the code coverage report on a server. For example, i have a jboss server application running in a QA environment and i’m running my tests from my laptop and the test is posting requests to the QA server. How would i get the code coverage on that server app?

    Best,
    Manu

    Reply
    • I would do this by following these steps:

      1. Attach Java agent to the used application server.
      2. Create code coverage report from the execution coverage data recorded by the Java agent.

      This should do the trick (in theory). I have never done this myself though.

      Reply
      • I have one more question on the agent. If i run the jacoco agent on an app server for long time and generate the code coverage report without reset whenever i need it, would it likely to cause any out of memory error in app server? Does the tcp server option in jacoco agent save the metrics locally or does it keep all the information in memory which can cause out of memory error?

        Reply
        • The JaCoCo documentation says that:

          The JaCoCo agent collects execution information and dumps it on request or when the JVM exits.

          I have no idea about the inner workings of the JaCoCo agent but I would assume that it keeps the collected information in memory.

          The documentation also says that if you run the agent in the file system mode, it writes the data to a file when JVM is terminated. Like you said, you can also run the agent in a mode where the data is written to a socket connection when it is requested (socket server or socket client modes) but I have no idea where the data is stored in this case. I would assume that it is stored in memory because the documentation doesn’t say otherwise.

          You could try to get a better answer to your question from the Google group called JaCoCo and EclEmma Users.

          Reply
  • Hi Petri,
    I get a question and need your help. As I was trying to collect report,I encountered trouble in filtering class file which I’m not interested in. Do you have any suggestion on how this filtering process can be set through maven plugin?In your case,is it set to not execute unit test case that you are uninterested ?

    Reply
    • Hi Kingsley,

      You can exclude classes by configuring a list of class files which are excluded from the report.

      Actually this week I had to exclude all JPA meta model classes from my unit test code coverage report (these are tested in integration tests). I did this by modifying the configuration of the execution which runs the report goal of the JaCoCo Maven plugin. My new configuration looks as follows:

      
      <execution>
          <id>post-unit-test</id>
          <phase>test</phase>
          <goals>
              <goal>report</goal>
          </goals>
          <configuration>
              <dataFile>
                  ${project.build.directory}/coverage-reports/jacoco-ut.exec
              </dataFile>
              <!-- Exclude the JPA meta model classes from the report -->
              <excludes>
                  <exclude>**/*_.class</exclude>
              </excludes>
              <outputDirectory>
                  ${project.reporting.outputDirectory}/jacoco-ut
              </outputDirectory>
          </configuration>
      </execution>
      
      

      I hope that this answered to your question.

      Reply
      • thanks for the reply ~

        Reply
  • Normally I do not read post on blogs, however I would like to say that this write-up very compelled me to check out and do it! Your writing style has been amazed me. Thank you, quite great post.

    Reply
    • Thank you for your kind words. I really appreciate them!

      Reply
  • Hi Petri,

    Question: How to get code coverage using the jacoco coverage engine while executing the robot acceptance test cases? Usually robot test cases are run after the UT and IT. How do I integrate the jacoco in such case. Really appreciate for any input.

    Thanks
    Greg

    Reply
    • Hi Greg,

      how do you run the robot test cases? Are you using the Robot Framework Maven plugin? I think that you have to

      1. Ensure that the JaCoCo runtime agent is running and gathering execution data when the robot test cases are executed.
      2. Create the code coverage report after the robot test cases haven been executed.

      Unfortunately this was the easy the part. The hard part is that I have no idea how you can accomplish this.

      I took a quick look at the documentation of the plugin’s run goal and I couldn’t find a way to pass custom arguments to the plugin (maybe the argumentFile configuration element is used for this purpose?) If it is, you could try to add the following line to that file (change the directory paths):

      -javaagent:/Users/loke/.m2/repository/org/jacoco/org.jacoco.agent/0.6.3.201306030806/org.jacoco.agent-0.6.3.201306030806-runtime.jar=destfile=/Users/loke/Projects/Java/Blog/maven-examples/code-coverage-jacoco/target/coverage-reports/jacoco-ut.exec

      This starts the JaCoCo runtime agent and configures the location of the file which contains the execution data.

      The second step is more problematic. You have to trigger the report goal of the JaCoCo Maven plugin after your robot test cases have been executed. Maybe you could configure the JaCoCo Maven plugin to invoke this goal when you create site for your project or something like this? I know it isn’t the optimal way to do this but at the moment I cannot figure out a better solution. :(

      Reply
  • Thanks for this — I was able to quickly add JaCoCo to my POM. Unfortunately, JaCoCo is incompatible with PowerMock’s byte code rewriting (https://github.com/jacoco/eclemma/issues/15). Luckily, Cobertura Maven plugin 2.6 works with Java 7.

    Reply
  • Thanks for this , however, when I ran mvn clean verify -P integration-test or mvn clean verify -P all-tests, the jacoco-it folder has index.html but shows 0% test coverage.
    My project code is under – src/main/java and my tests are under src/test/java
    can you please help?

    Reply
    • The tests should be placed as follows:

      • Integration tests must be placed to the src/integration-test/java directory.
      • Unit tests must be placed to the src/test/java directory.

      The test coverage is zero percent because you try to generate a code coverage report for integration tests but your tests are in the “wrong” directory.

      You can solve this problem in two ways:

      • Move the tests to the src/integration-test/java directory. You should do this only if the tests are integration tests and not unit tests.
      • Create the code coverage report by using the command mvn clean test. You should this only if the tests are unit tests.
      Reply
      • I copied all my integrations tests to src/integration-test/java directory but that too didnt help.
        I dont need code coverage report for the unit tests because we dont have any.
        Tests run: 13, Failures: 0, Errors: 0, Skipped: 0
        [INFO]
        [INFO] — jacoco-maven-plugin:0.6.3.201306030806:report (post-integration-test) @ #####
        [INFO] — maven-failsafe-plugin:2.16:verify (integration-tests) @ ######
        [INFO] ————————————————————————
        [INFO] BUILD SUCCESS
        [INFO] ————————————————————————

        the report still shows as 0% code coverage for my integration tests.

        Please help!

        Reply
        • Does the tested code run in the same JVM than the test, or are you testing an application which is deployed to a server?

          If the code is deployed to an application server, I would follow the steps given in this answer (I haven’t tried this myself so I have no idea if it works).

          If the tested code runs in the same JVM than the test, I probably need to your pom.xml so that I can figure out what is wrong. You can paste it to pastebin.com and add a link to your comment (don’t paste it here because Wordpress eats XML tags and the remaining text is pretty much useless).

          Reply
          • am not sure about the same JVM question that you asked..
            our application code and our integration tests are under the same folder structure…so when I run the mvn clean verify -P integration-test command on the terminal, the tests run against the code/service running on localhost .
            the tests run good, but the report shows 0% coverage… :-(

          • this is what I do:
            start my service locally, run my integration tests using mvn clean verify -P integration-test and then look into the target/jacoco-it folder for the report
            folder structure:

            I want to view only the integration test results , so i have used your stuff under the— Configuring The Code Coverage Report for Integration Tests
            can you please assist further?

          • Hi Raj,

            If you start the tested service manually, it is probably running in a different JVM than the integration tests (and the JaCoCo agent). That is why the code coverage is zero percent.

            You can fix this by starting the JaCoCo agent in the JVM which runs the tested code. If you use an application server, you can follow the instructions given in this comment (I haven’t done this myself I have no idea if this really works).

  • Thanks !! I was looking to add code coverage for integration tests. Jacoco and your blog was a perfect match !!

    Reply
    • You are welcome! I am happy to hear that this blog post was useful to you.

      Reply
  • thanks a lot. it was a wonderful article and did help me.
    I learnt a lot. thanks for all your efforts.

    Cheers
    Atul

    Reply
    • You are welcome. I am happy to hear that this blog post was useful to you.

      Reply
  • Petri,

    Thanks for your blog. I was looking for Integration test code coverage and your blog is on spot on what I was looking for.

    However just out of interest, are you aware of any tool which does code coverage at run time? For example, when we run automated service test packs with Soap UI tool or such tools against some web service, is it possible to find out the number of lines, methods or business blocks executed at service during run time? If we have anything of this sort, It would help to see if we have enough service test packs to cover all scenarios and also it would help to get rid of redundant code which is never used.

    Any idea on this?

    Thanks
    Rajesh

    Reply
    • Hi Rajesh,

      I haven’t tried this myself but it should be possible (at least in theory).

      By the way, thank you for posting this comment. I have never thought about using JaCoCo in this way, and this is actually a pretty good way to identify unused code which can be deleted.

      Reply
  • Thanks a million.
    This is much better documentation than jacoco’s official docs.

    Reply
    • Thank you! I am happy to hear that this blog post was useful to you.

      Reply
  • Hi Petri,

    thank you very much for this useful post!

    I have an issue about excluding classes from JaCoCo reports. I copy-pasted you configuration for
    unit test coverage (don’t have any ITs) only extra thing I added to configuration of both executions is

    *MersenneTwisterFast*

    The exclude for jacoco:coverage works fine but the MersenneTwisterFast class is still shown in reports with 0% coverage. Any idea why it does not work? I searched the web and only thing I found was people missing out the exclude for reports, but I do have it there.

    Reply
    • Solved,

      jacoco:coverage

      **/MersenneTwisterFast.java

      jacoco:report

      **/MersenneTwisterFast.class

      Sorry for bothering you.
      Keep up the good work!

      Reply
      • Tomas,

        It is good to hear that you were able to solve your problem! And no worries, feel free to ask questions if you have problems or if you cannot understand something. Unfortunately I couldn’t help you this time since I was outside chopping firewood.

        Reply
  • Hi Petri

    Excellent article. Your way of writing is very concise and clear. I have one question.
    I am trying to build a framework for my project. We use restful web services with json.
    Now this is what I want to do.

    1. Create a test database. There I will store testcase id, target url, json request, expected response etc. This can be more elaborate and exhaustive if required.
    2. When I start to run my test suite, there will be a component which will read all test cases from database.
    3. It will fire each test case and collect actual response and will update a table with actual response.
    4. Finally there will be another component which will read this table and prepare an html report indicating total test cases run, failed and passed test cases.

    What I would like to know is, are there any issues with this approach?
    Second, is there any way if, after building this framework, I can get coverage percentage along with failed and passed test cases? Will the plugin which you described above help me get the coverage percentage?

    The scenario I am describing would happen on QA environment with servers running.
    Appreciate if you would let me know your thoughts.

    Thanks for this wonderful article again.
    Cheers

    Reply
    • What I would like to know is, are there any issues with this approach?

      Well, it seems that you have to write a lot of plumbing code to get this done, but this might be a reasonable approach if you want to create new tests without writing any code. I have never done anything similar myself so unfortunately I cannot give any specific advice. Are you planning to extend an existing framework (such as Cucumber or Robot), or do you want write your own framework?

      Is there any way if, after building this framework, I can get coverage percentage along with failed and passed test cases?

      To be honest, I don’t know the answer to this question because I don’t know if JaCoCo can differentiate the failed and passed test cases. I assume that it cannot do this because it cannot know whether your test case fails or passes.

      Will the plugin which you described above help me get the coverage percentage?

      It can help you to get the code coverage percentage but since the JaCoCo agent must be run in the server VM, you would have to download the execution data files from the server and figure out a way to generate the coverage report after the files have been downloaded. I think that this is doable but since I have never done this myself, I cannot give you any specific advice.

      Reply
  • Hi Petri,

    Thanks for the awesome write-up. Basically, I am working on a program/tool which would invoke use JaCoCo on a package to generate reports and use those generated reports to get selected fields from it and make a whole new report. I know this sounds redundant but I have to do it anyways. I haven’t yet started working on it yet as I am thinking of quick and efficient ways of doing this. Since I am a complete beginner to this area, I wanted some ideas/suggestions from you. Can you help me out with this? Thanks.

    Reply
    • There are a few things that come to my mind:

      • I assume that your tool would create those reports from the execution metadata gathered by the JaCoCo agent? The reason why I ask this is that the JaCoCo agent must be running in the same VM than the “inspected” code.
      • Do you think that you could use the the report goal of the JaCoCo Maven plugin when you create the first report? This way you wouldn’t have to read the execution metadata yourself (I tried to find its specification from the Google but I couldn’t find anything). After the first report is created, you can find the information you need and create the another report.

      I know that this isn’t much, but I hope that it helps you to get started.

      Reply
  • Thanks for this wonderful article

    JaCoCo not consider spring data interface repository to calculate coverage metrics
    How tell jacoco to cover Spring data repository integration test ?

    Thanks

    Reply
    • The JaCoCo FAQ states that:

      Java interface methods do not contain code, therefore code coverage cannot be evaluated. Indeed code coverage is recorded for the implementation classes. The same applies to abstract methods in abstract classes.

      It seems that there is no way to include repository interfaces in your code coverage report :(

      Reply
      • Thanks
        For now I added in ineterfaces constants that contains test values ​​for the input parameters.
        This is not propore!

        Reply
  • Hi Petri,
    Excellent article.
    I am trying to get the integration coverage report using jacoco for my selenium web test cases which is a seperate testcases module(using failsafe plugin). Application is deployed in remote tomcat server. I am executing the test cases from another server. How can i get the IT coverage using maven plugins.

    Thanks
    Shankar

    Reply
    • Hi,

      I proposed one possible solution to your problem in this comment. Unfortunately I haven’t done this myself so I am not sure how you can get the execution data from the remote server before you create the code coverage report.

      Reply
      • Hi,
        Thanks,
        I have added the java agent to my tomcat JAVA_OPTS where my application is installed
        Once i start the tomcat , jacoco.exec is created. Also i have the classdumpdir to the java_opts.
        After doing some operation , classdumpdir has dumped the classes but the jacoco.exec has nothing.
        So my coverage report is Zero

        Thanks

        Reply
        • It seems that I should investigate this problem and write a blog post about it. I added this blog post idea to my Trello board, and I will write this blog post after I have finished a few other blog posts that I need to write before this one.

          Reply
  • Hi Petri,
    Can you give me some guideline in integrating cobertura and Fitnesse through Maven?

    Reply

Leave a Comment