Integration Testing With Maven

Adding integration tests to a Maven build has traditionally been a bit painful. I suspect that the reason for this is that the standard directory layout has only one test directory (src/test).

If we want to use the standard directory layout and add integration tests to our Maven build, we have two options:

First, we can add our integration tests to the same directory as our unit tests. This is an awful idea because integration tests and unit tests are totally different beasts and this approach forces us to mix them. Also, if we follow this approach, running unit tests from our IDE becomes a pain in the ass. When we run tests, our IDE runs all tests found from the test directory. This means that both unit and integration tests are run. If we are “lucky”, this means that our test suite is slower than it could be, but often this means that our integration tests fail every time. Not nice, huh?

Second, we can add our integration tests to a new module. This is overkill because it forces us to transform our project into a multi-module project only because we want to separate our integration tests from our unit tests. Also, if our project is already a multi-module project, and we want to write integration tests for more than one module, we are screwed. Of course we can always create a separate integration test module for each tested module, but it would be less painful to shoot ourselves in the foot.

It is pretty clear that both of these solutions suck. This blog post describes how we can solve the problems of these solutions.

The requirements of our Maven build are:

  • Integration and unit tests must have separate source directories. The src/integration-test/java directory must contain the source code of our integration tests and the src/test/java directory must contain the source code of our unit tests.
  • Integration and unit tests must have different resource directories. The src/integration-test/resources directory must contain the resources of our integration tests and the src/test/resources directory must contain the resources of our unit tests.
  • Only unit tests are run by default.
  • It must be possible to run only integration tests.
  • If an integration test fails, it must fail our build.
  • The name of each integration test class must start with the prefix 'IT'.

Let’s start by creating Maven profiles for unit and integration tests.

Creating Maven Profiles for Unit and Integration Tests

First, we need to create two Maven profiles that are described in the following:

The dev profile is used in the development environment, and it is the default profile of our Maven build (i.e. It is active when the active profile is not specified). This Maven profile has two goals that are described in the following:

  1. It configures the name of the directory that contains the properties file which contains the configuration used in the development environment.
  2. It ensures that only unit tests are run when the dev profile is active.

The integration-test profile is used for running the integration tests. The integration-test profile has two goals that are described in the following:

  1. It configures the name of the directory that contains the properties file which contains the configuration used by our integration tests.
  2. It ensures that only integration tests are run when the integration-test profile is active.

We can create these profiles by adding the following XML to our pom.xml file:

<profiles>
    <!-- The Configuration of the development profile -->
    <profile>
        <id>dev</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <properties>
            <!--
                Specifies the build.profile.id property that must be equal than the name of
                the directory that contains the profile specific configuration file.
                Because the name of the directory that contains the configuration file of the
                development profile is dev, we must set the value of the build.profile.id
                property to dev.
            -->
            <build.profile.id>dev</build.profile.id>
            <!--
                Only unit tests are run when the development profile is active
            -->
            <skip.integration.tests>true</skip.integration.tests>
            <skip.unit.tests>false</skip.unit.tests>
        </properties>
    </profile>
    <!-- The Configuration of the integration-test profile -->
    <profile>
        <id>integration-test</id>
        <properties>
            <!--
                Specifies the build.profile.id property that must be equal than the name of
                the directory that contains the profile specific configuration file.
                Because the name of the directory that contains the configuration file of the
                integration-test profile is integration-test, we must set the value of the
                build.profile.id property to integration-test.
            -->
            <build.profile.id>integration-test</build.profile.id>
            <!--
                Only integration tests are run when the integration-test profile is active
            -->
            <skip.integration.tests>false</skip.integration.tests>
            <skip.unit.tests>true</skip.unit.tests>
        </properties>
    </profile>
</profiles>

Second, we have to ensure that our application uses the correct configuration. When we run it by using the dev profile, we want that it uses the configuration that is used in the development environment. On the other hand, when we run our integration tests, we want that the tested code uses the configuration tailored for our integration tests.

We can ensure that our application uses the profile specific configuration by following these steps:

  1. Configure our Maven build to load the profile specific configuration file (config.properties) from the configuration directory of the active Maven profile.
  2. Configure the resource directory of our build (src/main/resources) and enable resources filtering. This ensures that the placeholders found from our resources are replaced with the actual property values that are read from the profile specific configuration file.

The relevant part of our POM file looks as follows:

<filters>
    <!--
        Ensures that the config.properties file is always loaded from the
        configuration directory of the active Maven profile.
    -->
    <filter>profiles/${build.profile.id}/config.properties</filter>
</filters>
<resources>
    <!--
        Placeholders that are found from the files located in the configured resource
        directories are replaced with the property values found from the profile
        specific configuration file.
    -->
    <resource>
        <filtering>true</filtering>
        <directory>src/main/resources</directory>
    </resource>
</resources>

Let’s move on and find out how we can add extra source and resource directories to our Maven build.

Adding Extra Source and Resource Directories to Our Maven Build

The requirements of our Maven build state that our integration and unit tests must have separate source and resource directories. Because the source code and the resources of our unit tests are found from the standard test directories, we have to create new source and resource directories for our integration tests.

We can add extra source and resource directories to our build by using the Build Helper Maven Plugin. We fulfill our requirements by following these steps:

  1. Add the Build Helper Maven Plugin to the plugins section of our POM file.
  2. Configure the executions that add the new source and resource directories to our build.

First, we can add the Build Helper Maven Plugin to our Maven build by adding the following XML to the plugins section of our pom.xml file:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>1.9.1</version>
    <executions>
        <!-- Add executions here -->
    </executions>
</plugin>

Second, we have to configure the executions that add our new source and resource directories to our Maven build. We can do this by following these steps:

  1. Create an execution that adds the new source directory (src/integration-test/java) to our build.
  2. Create an execution that adds the new resource directory (src/integration-test/resources) to our build and enable resource filtering.

We can create these executions by adding the following XML to the configuration of the Build Helper Maven Plugin:

<!-- Add a new source directory to our build -->
<execution>
    <id>add-integration-test-sources</id>
    <phase>generate-test-sources</phase>
    <goals>
        <goal>add-test-source</goal>
    </goals>
    <configuration>
        <!-- Configures the source directory of our integration tests -->
        <sources>
            <source>src/integration-test/java</source>
        </sources>
    </configuration>
</execution>
<!-- Add a new resource directory to our build -->
<execution>
	<id>add-integration-test-resources</id>
	<phase>generate-test-resources</phase>
	<goals>
    	<goal>add-test-resource</goal>
	</goals>
	<configuration>
    	<!-- Configures the resource directory of our integration tests -->
    	<resources>
        	<!--
            	Placeholders that are found from the files located in the configured resource
            	directories are replaced with the property values found from the profile
            	specific configuration file.
        	-->
        	<resource>
            	<filtering>true</filtering>
            	<directory>src/integration-test/resources</directory>
        	</resource>
		</resources>
	</configuration>
</execution>

Let’s find out how we can run our unit tests with Maven.

Running Unit Tests

If we think about the requirements of our Maven build, we notice that we must take the following things into account when we run our unit tests:

  • Unit tests are run only when the dev profile is active.
  • When we run our unit tests, our build must run all test methods found from the test classes whose name don’t start with the prefix 'IT'.

We can run our unit tests by using the Maven Surefire plugin. We can configure it by following this steps:

  1. Configure the plugin to skip unit tests if the value of the skip.unit.tests property is true.
  2. Configure the plugin to ignore our integration tests.

We can configure the Maven Surefire plugin by adding the following XML to the plugins section of our POM file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18</version>
    <configuration>
        <!-- Skips unit tests if the value of skip.unit.tests property is true -->
        <skipTests>${skip.unit.tests}</skipTests>
        <!-- Excludes integration tests when unit tests are run -->
        <excludes>
            <exclude>**/IT*.java</exclude>
        </excludes>
    </configuration>
</plugin>

We can run our unit tests by using the command: mvn clean test -P dev.

Let's move on and find out how we can run our integration tests with Maven.

Running Integration Tests

If we think about the requirements of our Maven build, we notice that we have to take the following things into account when we run our integration tests:

  • Integration tests are run only when the integration-test profile is active.
  • If an integration test fails, our build must fail as well.
  • When we run our integration tests, our build must run all test methods found from the test classes whose name starts with the prefix 'IT'.

We can run our integration tests by using the Maven Failsafe plugin. We can configure it by following these steps:

  1. Create an execution that invokes the integration-test and verify goals of the Maven Failsafe plugin when we run our integration tests. The integration-test goal runs our integration tests in the integration-test phase of the Maven default lifecycle. The verify goal verifies the results of our integration test suite and fails the build if it finds failed integration tests.
  2. Configure the execution to skip integration tests if the value of the skip.integration.tests property is true.

We can configure the Maven Failsafe Plugin by adding the following XML to the plugins section of our POM file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.18</version>
    <executions>
        <!--
            Invokes both the integration-test and the verify goals of the
            Failsafe Maven plugin
        -->
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
            <configuration>
                <!--
                    Skips integration tests if the value of skip.integration.tests
                    property is true
                -->
                <skipTests>${skip.integration.tests}</skipTests>
            </configuration>
        </execution>
    </executions>
</plugin>

We can run our integration tests by using the command: mvn clean verify -P integration-test.

Let's move on and summarize what we learned from this blog post.

Summary

This blog post has taught us three things:

  • We can add extra source or resource directories to our Maven build by using the Build Helper Maven Plugin.
  • We can run our unit tests by using the Maven Surefire plugin.
  • We can run our integration tests by using the Maven Failsafe plugin.
89 comments… add one
  • Petri Oct 23, 2012 @ 16:33

    If we are writing integration tests for real life applications, we often have to use integration test specific resource files (E.g. DBUnit dataset files). We can do this by ensuring that the add-test-resource goal of Builder Helper Maven plugin is executed at Maven’s generate-test-resources lifecycle phase.

    We do this by adding the following snippet to the executions section of the Builder Helper Maven plugin configuration:

    <execution>
    	<id>add-integration-test-resources</id>
    	<phase>generate-test-resources</phase>
    	<goals>
    		<goal>add-test-resource</goal>
    	</goals>
    	<configuration>
    		<resources>
    			<resource>
    				<directory>src/integration-test/resources</directory>
    			</resource>
    		</resources>
    	</configuration>
    </execution>
    

    This ensures that the directory src/integration-test/resources is added to our Maven build.

    • Stephen Hartley Oct 2, 2015 @ 1:07

      Great post, very helpful thanks.
      I have followed your advice and added src/integration-test/resources as a test resource as above. However, this causes the integration test resources to be included in the production artifact (jar). The integration sources are correctly excluded, so I don't understand why the resources are not handled the same way. Do you have any ideas? I also asked this question on stackoverflow: http://stackoverflow.com/questions/32885696

      • Petri Oct 2, 2015 @ 20:01

        Hi Stephen,

        When I packaged the example application of this blog post, integration test resources weren't added to the created jar file. Have you modified the configuration of the Maven Jar plugin?

        • Stephen Hartley Oct 2, 2015 @ 20:27

          Hmm, will have to check this again as I'm using the maven shade plugin to build an "uber jar". Perhaps this is causing problems...

          Thanks again!

          • Petri Oct 2, 2015 @ 22:44

            Hi Stephen,

            That can definitely be a problem, but I noticed that there is a way to exclude files from the "uber jar". Can you exclude the integration test resources from it?

    • Viktoriya Aug 12, 2018 @ 12:50

      Hi)) I have a long time trying to figure out how to learn to write integration tests, but I do not succeed. I find a lot of articles on this topic, but my head was a mess - all mixed up. Please help.

      • Petri Aug 12, 2018 @ 14:21

        Hi,

        What is the biggest problem that prevents you from achieving your goal? Also, what framework (Spring, Java EE, and so on) are you using?

  • Celso Marques Feb 11, 2013 @ 18:30

    Hey Petri,

    Very nice article.
    It was very helpful for me!!!
    Thanks a lot.

    Best Regards,
    Celso

    • Petri Feb 11, 2013 @ 20:35

      You are welcome! I am happy that I could help you out.

  • rang Aug 29, 2013 @ 1:44

    Hi Petri,

    I want to use Jacoco for web application code coverage.
    I think the above only works fine if you are running tests and Web Server on the same machine.

    Currently my Jenkins run on one machine and my server is on another machines.Does it still work for that scenarior or what changes should I do for that.

    • Petri Aug 30, 2013 @ 8:57

      Hi Rang,

      If you are referring to this blog post, I have to confess that I have not tried running the example with Jenkins yet.

      I am planning to do this at some point because I want to make sure that it works. In the meantime, you might want to take a look at this blog post which describes the usage of the JaCoCo Jenkins plugin.

    • Walt Sep 12, 2013 @ 4:20

      Jacoco has a tcpmode option that can operate as either client or server. From the Jenkins CI - you can pull the coverage metrics and reset the collection. So as long as you start the server with the appropriate agent configuration you will be able to collect coverage data remotely (be advised ther is no security here - so one would only do this in a trusted environment)

      • Petri Sep 15, 2013 @ 17:14

        Thanks for the tip! I will try this out when I have got some time to do it and write a new blog post about it.

  • Declan Cox Oct 3, 2013 @ 18:35

    Very helpful post. Thanks.

    • Petri Oct 3, 2013 @ 18:57

      You are welcome!

  • Chad Showalter Oct 21, 2013 @ 18:09

    It would sure be nice if we didn't have the magic name requirement (**/IT*.java) for excluding integration tests from the unit test build. Do you know of a way to accomplish this?

    • Petri Oct 21, 2013 @ 22:42

      Yes, that would be nice.

      One option is to move the configuration of the Build Helper Maven plugin to the profile which you use to run the integration tests. This way the integration tests would be added to the classpath only when you use the correct profile which means that you can name your tests anyway you want.

      I haven't tried this out but it should work (in theory).

      • Brooke Smith Jul 14, 2016 @ 3:40

        The trouble with this is that maven includes `src/test/java` by default and there seems no way of preventing this. So `**/IT*java` or similar is required. After having used Gradle and returning to Maven, I'm understanding why many devs don't like Maven.

  • Emiliano Feb 7, 2014 @ 2:49

    Hello, great contribution Petri.
    I used this configuration in a multi module project and it worked ok.
    The only thing is that using Eclipse (Kepler + Maven 3) I found that the eclipse plugin had a problem with the execution declaration in the maven helper plugin.
    If this happens to anyone else, here are the three possible solutions:
    1. In the quick fix (of eclipse) select the option of adding the ignore of the execution to the eclipse build in the eclipse project file. Even though it warns that it´s in an experimental fase, it works fine.
    2. In the quick fix (of eclipse) select the option of adding the ignore to the eclipse build in the project POM.
    3. Update the m2 lifecycle mapping connector. It connects to the marketplace and installs it. Restart eclipse and the problem is solved.

    These three approaches are independent.

    Hope this help to somebody.

    • Petri Feb 7, 2014 @ 10:31

      Hi Emiliano,

      It is nice to know that this tutorial was useful to you. Also, thank you for sharing the solutions for the problem you had with Eclipse and Build Helper Maven plugin. I am sure that this information is useful to someone!

    • SilencerBY Jul 8, 2014 @ 11:17

      Approaches 1 & 2 deprive you of a properly setup .classpath (i.e. src/integration-test/java appears as a source folder in Eclipse).

      BTW For the latest version of 'm2e connector for build-helper-maven-plugin' see this thread http://comments.gmane.org/gmane.comp.ide.eclipse.plugins.m2eclipse.user/10082

      • Petri Jul 8, 2014 @ 16:06

        Thank you for leaving this comment! I am sure that it is useful for Eclipse users.

  • Tej Feb 19, 2014 @ 11:07

    Hello Petri,
    Very informative and nice article.

    Need suggestion: I am planning to write integration test for my product.

    Product have 3 different web applications with 3 different spring configuration. Currently, Integration test is written to load just base configuration but I want to separate modules(applications) and write common integration test across modules ..so planing to create 4 modules like one for base and rest 3 like application specific modules so each specific module will load their own spring configuration while doing integration test.

    What do you think will be best approach? Your help and views will be greatly appreciated.
    Many Thanks

    • Petri Feb 19, 2014 @ 20:24

      Hi Tej,

      I would configure the application context in each web application module (no configuration in the core module). The reason for this is that if you have to override the base configuration, you can run into really weird problems. I have to admit that my approach means that you probably have to create similar application context configuration classes / files in each web application module but I am ready to make that tradeoff.

      Also, I would simply write integration tests for each web application. This way I could ensure that the base module is working correctly with each web application.

      I hope that this answered to your question. If you have any additional questions, feel free to ask them!

  • Sachith Mar 26, 2014 @ 16:29

    This is by far one of the best posts I've read on Integration testing. The content is very well presented too.

    Keep up the great work.

    • Petri Mar 27, 2014 @ 9:47

      Thanks! I am happy to hear that this blog post was useful to you.

  • Afonso Mar 28, 2014 @ 12:32

    Greetings,
    Very nice article! However, I am not quite being able to pull this off.
    "mvn clean verify -P integration-test" is correctly executing only the TestOneIT.java I have in my src/it/java folder

    However, the "mvn clean test " is executing my jUnit tests inside src/test/java AND my TestOneIt.java as well :( Like the if the ${skip.integration.tests}
    inside the maven-failsafe-plugin isn't doing nothing.

    Help?

    • Afonso Mar 28, 2014 @ 12:55

      Sorry, my bad. the problem was the search pattern for the integration test files.
      It was just a matter of changing **/IT*.java to **/*IT*.java

      • Petri Mar 28, 2014 @ 21:02

        No worries. Good to hear that you were able to solve your problem!

  • r.k May 1, 2014 @ 10:03

    good one :)

    • Petri May 1, 2014 @ 18:08

      Thanks! I am happy to hear that you liked this blog post.

  • Craig May 15, 2014 @ 22:05

    Hi Petri, I'm working in a multi-module setting, I have:

    pom.xml
    - persistence
    - app-services
    - rest

    Each of these modules have both unit and integration tests, and I was wondering if I can just declare all of these profile settings in the parent pom? The reason I'm not sure is because of the filtering, the parent won't have any resources to filter (it's just a "pom")

    I know this is a candidate for TIAS, but I figured if you knew off the top of your head it would save me time mocking up a bunch of tests.

  • Marco Jul 2, 2014 @ 8:23

    Hi Petri and thank you for your very helpful tutorial!

    I'm trying to implement integration-testing in the way you describe. In my situation I also need to keep test .xml context configuration files separated by ones of application. So I put them into /src/integration-test/resources/META-INF.

    Looking in target folder I have found that: files in src/main/resources are placed under PROJECT/target/classes/, while files in /src/integration-test/resources/ are placed in PROJECT/target/test-classes

    application.properties in PROJECT/target/classes is processed by maven plugins that replace ${placeholder} with value of ${profile}/config.properties, while application.properties in PROJECT/target/test-classes is not processed. So executing maven integration-test I get exception like: Circular placeholder reference 'db.driver' in property definitions.

    Trying to put application.properties without placeholders in src/integration-test/resources seems than application.properties is not loaded by @PropertySource("classpath:application.properties"). I get exception like: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set.

    So I'm wrong in setting my test context, or I need to modify my pom to reach my requirements?

    Another question is, what if I want to execute/debug each single test from eclipse?

    Thank you very much Petri!

    Have a good day

    • Petri Jul 2, 2014 @ 11:04

      Hi Marco,

      The idea behind the configuration explained here is that you can use the same properties file (src/main/resources/application.properties) in your application and in your tests because Maven selects the actual property values based on the active profile.

      In other words,

      • If the active profile is dev the placeholders found from the src/main/resources/application.properties file are replaced with the property values found from the profiles/dev/config.properties file.
      • If the active profile is integration-test the placeholders found from the src/main/resources/application.properties file are replaced with the property values found from the profiles/integration-test/config.properties file.

      This means that you don't need to create a separate properties file to the src/integration-test/resources directory because you can use the properties file found from the src/main/resources directory. This file contains the test specific configuration as long as the correct profile (integration-test) is active.

      You can the refer to this file in your test specific application context configuration file like this: 'classpath:application.properties'.

      On the other hand, if you want to create a separate properties file for your integration tests, you shouldn't name it application.properties because if you do, the classpath has two files called application.properties and this can cause problems if you refer to them by using the notation: 'classpath:application.properties' (the classloader loads one of them but you cannot know which one).

      I hope that this shed some light to your properties file related problems.

      I haven't used Eclipse for six years but if I have to run / debug a single integration test in my IDE (IntelliJ Idea), I follow these steps:

      1. Compile the project when the integration-test profile is active. This is required so that the correct property values are filtered to the application.properties file.
      2. Run the test in IDE. Typically I can just click the name of the test method with the right mouse button and select 'Run test'. I assume that Eclipse is working in a similar way.

      I hope that this answered to your questions. If not, let me know!

      • Marco Jul 3, 2014 @ 8:10

        Thank you Petri for your reply!

        I have tryed to use /src/main/resources/application.properties in conjunction with .xml configuration files in /src/intergation-test/resources/META-INF/*-test.xml (I have renamed them with test suffix) but context initialization fails: placeholders cannot be found.

        Here is how test is configured:

        
        @RunWith(SpringJUnit4ClassRunner.class)
        @PropertySource("classpath:application.properties")
        @ContextConfiguration(locations={
        		"classpath:/META-INF/datasource_test.xml",
        		"classpath:/META-INF/jpa-tx-config_test.xml"
        		})
        @TestExecutionListeners({ DependencyInjectionTestExecutionListener.class,
        	DirtiesContextTestExecutionListener.class,
        	TransactionalTestExecutionListener.class,
        	DbUnitTestExecutionListener.class })
        @DirtiesContext(classMode=ClassMode.AFTER_EACH_TEST_METHOD)
        @ActiveProfiles("dev")
        
        

        So application.properties can be found on target/classes, with other src/main/resources configuration files, while test configuration files can be found on target/test-classes.

        Seems that if application.properties is not in same folder of xmls, placeholders cannot be resolved..

        • Petri Jul 3, 2014 @ 10:13

          Hi Marco,

          I noticed that you use the @PropertySource annotation in your test class. I assume that you are trying to load the properties file which contains the property values used in your context configuration files. Am I correct?

          The thing is that the javadoc of the @PropertySource annotation states that it works only if it is used together with the @Configuration annotation. In other words, you should use Java configuration.

          If you want to use XML configuration, you can configure the used properties file by using the property-placeholder tag of the context namespace.

          You could also verify that the property values are found from the application.properties file after it has been copied to the target/classes directory. If this is the case, you know that the problem isn't in your build script. If the values are not found from this file, the problem is that Maven isn't filtering the property values from your profile specific configuration files to the application.properties file.

          By the way, if you cannot solve this problem, it would be helpful to see your build script / project. Is this possible or is it a work related project?

          • Marco Jul 3, 2014 @ 13:19

            Thank you Petri, you give me the right hint.

            Just enable in xml do the trick! So I hare removed @PropertySource annotation (that as documentation states works only if it is used together with the @Configuration annotation).

            It also works using xml files in src/integration-testing/resources, preferred over that ones defined in src/main/resources

            Thank you very much Petri!! Have a nice day

          • Marco Jul 4, 2014 @ 15:23

            It also works using xml files in src/integration-testing/resources, preferred over that ones defined in src/main/resources: not true because when I run application from my eclipse workspace xml files conflict.. but I have solved appending a suffix to each xml file.

          • Petri Jul 4, 2014 @ 16:27

            Hi,

            It is good to hear that you were able to solve your problem.

            I have always used the same XML configuration files for every profile (dev, test, prod, and integration-test), and moved the configuration to profile specific configuration files (although nowadays I use only Java configuration).

            On the other hand, this might not always be good enough because you might want to use different beans when you run your integration tests. I have solved this problem by using bean definition profiles (aka Spring profiles).

  • Tom Aug 25, 2014 @ 12:06

    Hi Petri,
    This solved the problem I was having, very good post, thanks a lot.
    Tip: add a bonus to this entry and show how to run tests in parellel, it's very handy and explained here http://maven.apache.org/surefire/maven-failsafe-plugin/examples/junit.html.

    Cheers
    Tom

    • Petri Aug 25, 2014 @ 20:33

      Hi Tom,

      I am happy to hear that this blog post was useful. Also, I will update this blog post when I have got time to do it and include the tip you mentioned in it. Thanks for sharing.

  • Srinivas Aug 25, 2014 @ 15:13

    Thanks a loooooot!!!
    you made my day!!!

    • Petri Aug 25, 2014 @ 20:16

      You are welcome! I am happy to hear that this blog post was useful to you.

  • Krishna Singh Oct 14, 2014 @ 12:03

    Dear Petri,
    I'm currently trying to generate code coverage reports Java Server for manual tests in a Maven project. I'm using eclipse with the JaCoCO plugin.

    The reports are properly generated when using automated Junit tests.

    So What I need is

    1. Run the program
    2. Perform some function on UI(Manual Testing)
    3. Exit the program
    4. Get a code coverage report for that run

    So can suggest me some way to achieve it.
    Thanks in advance.

    Best Regards,
    Krishna Singh

  • Anonymous Nov 26, 2014 @ 8:41

    Thanks a lot. It's very helpful for me!

    • Petri Nov 26, 2014 @ 8:56

      You are welcome! I am happy to hear that this blog post was useful to you.

  • Pradeep Kumar Dec 8, 2014 @ 16:30

    Hi,
    Thank you for this great article. I am completely new to this world but voila! i was able to work with the steps mentioned above. However I am facing 1 issue.
    maven while building is looking for config.properties under {basedir}\profiles\* than {basedir}\src\main\profiles.
    Any clue?

    • Petri Dec 8, 2014 @ 16:42

      If you want to change the location of the profiles directory, you have to change the value of the filter element.

      For example, if you want to put the profile specific configuration directories to the src/main/profiles directory, you have to use the following configuration:

      
      <filters>
          <filter>src/main/profiles/${build.profile.id}/config.properties</filter>
      </filters>
      
      

      If you want to get more information about this, you should read this blog post.

  • Fabrizio May 7, 2015 @ 15:50

    Excellent work! Many thanks Petri.

    • Petri May 7, 2015 @ 17:37

      You are welcome! I am happy to hear that this blog post was useful to you.

  • Fedor Jun 2, 2015 @ 13:20

    Thanks Petri for very useful article.

    I've a question on integration tests: I need to run the same integration tests several times on a web application in different configurations. For example, a web app recognises propertyA, I want to integration test the web app with propertyA=1234 and propertyA=4321.

    How would you recommend to do that?

    • Petri Jun 2, 2015 @ 23:16

      Hi Fedor,

      Thank you for your kind words. I really appreciate them.

      Do you think that you could create a controller method that is used to set the value of the propertyA before your integration tests are run? Of course you would have to ensure that this method can be invoked only when the integration tests are run, but this shouldn’t be a problem.

      I know that this sounds like a hack, but if you want to change the value of that property without using different Maven profiles for each legal property value, this is the easiest way to do it. If you find another way, let me know!

  • Markus Jun 3, 2015 @ 13:34

    Verify helpful and well written. Thanks!

    • Petri Jun 3, 2015 @ 17:34

      You are welcome! I am happy to hear that this blog post was useful to you.

  • Adam Jun 24, 2015 @ 16:42

    Hi Petri,
    thanks for the excellent article. Are you using JUnit 4.8? If so, there is a nice way to filter using a JUnit category and an annotation: http://www.java-tutorial.ch/maven/maven-integration-test

    I also decided to keep integration testing associated with the maven integration-test lifecycle phase. The only downside is that it runs the test phase first, but that can be inhibited using a variable like so:

    org.apache.maven.plugins
    maven-surefire-plugin
    2.17

    ${skip.unit.tests}
    com.my.junit.marker.IntegrationTest

    • Petri Jun 24, 2015 @ 23:37

      Hi Adam,

      Thank you for sharing this. Although I prefer keeping my integration and unit tests in different directories, the @Category annotation helps me to get rid of the stupid IT prefix (or suffix)!

      Also, I think that it might me help to implement a few other use cases as well. I have to take a closer look at it and figure out if I can create tests that are invoked only when the build is run on a CI server.

  • SetNug Mar 4, 2016 @ 14:53

    Hi Petri,

    Thank you for your tutorial.

    In my case I have OSGI plugin tests and unit tests in one OSGI fragment project, but I want to execute them separately using approach in your tutorial. So basically between tycho-surefire and maven-surefire plugin. Is it possible ?

    Thanks & Regards
    SetNug

  • Sourabh Mar 17, 2016 @ 13:36

    Hi Petri,

    Great article, it really helped a lot.

    I have more problem statement where we have two different standalone projects ( Project A and Project B )
    Project A is a application which has all the code and can be deployed on different servers.
    Project B have all the tests written which will test Project A

    I want to find a way such that i can find Code coverage for project A by running tests from project B. ( Both the projects are maven projects )

  • Richards Apr 14, 2016 @ 9:29

    Hi,

    I am relatively new to maven and TDD. I am reading the book - Growing Object Oriented Software Guided by Tests. The project explained in the book keeps unit tests, integration tests and end-to-end tests (all in java) in three different test-source directories. The project explained in the book uses Ant to build the project. I am exploring ways to accomplish the same in maven. That is how I ran into your post about integration testing with maven.

    I followed most of the items described in this post. But I would like to clarify the following items with you:
    1. Is it mandatory to create profile specific config files? Will they be used only when there are profile specific settings that need to be applied?
    2. If I want to add one more profile for end-to-end tests, use a separate test directory for them and run them independent of unit and integration tests, will I be able to achieve it using maven? If I can perform this task using maven, can you please give me some tips to accomplish it?

    Thanks,
    Richards.

    • Richards Apr 15, 2016 @ 6:09

      Hi,

      I think I should address my second query in a different manner. May be I should treat end-to-end tests as integration tests itself. I think I should look at the possibility of adding end-to-end tests into a separate test-source directory but allow them to execute along with integration tests. Is there a better approach?

      Thanks,
      Richards.

      • Petri Apr 15, 2016 @ 8:27

        Hi Richards,

        I am sorry that it took me some time to answer to your comment.

        Is it mandatory to create profile specific config files? Will they be used only when there are profile specific settings that need to be applied?

        It is not mandatory to create these configuration files, but if you decide to not create them, you shouldn't configure them in your POM file. Otherwise your build will fail because Maven cannot find the missing the configuration files.

        I think I should look at the possibility of adding end-to-end tests into a separate test-source directory but allow them to execute along with integration tests. Is there a better approach?

        You can definitely run end-to-end tests with Maven even if you have unit and integration tests as well. Your second question (running e-2-e tests together with integration tests) is quite interesting because I think that a good build script (and test suite) gives you the possibility to select the invoked tests. This is important because you want to run different tests in different situations.

        For example, if you are a developer, you might want to run only unit tests because integration and e-2-e tests are slow.

        On the other hand, if you are building the project on CI server, you can create different builds (every commit, daily etc) and select the invoked tests based on the build type. For example, it might no be practical to run all tests for every commit because getting the feedback takes too long.

        If you have any additional questions, don't hesitate to ask them.

        • Richards Apr 18, 2016 @ 12:17

          Hi Petri,

          Thank you for your reply.

          "You can definitely run end-to-end tests with Maven even if you have unit and integration tests as well."

          - If my end-to-end tests are also written in java (for a simple application) and stored under /src/java/e2e/ directory, can you please tell me how they can be executed independent of unit and integration tests? I am not sure how to configure maven-surefire-plugin and maven-failsafe-plugin to meet this requirement.

          Thanks,
          Richards.

          • Petri Apr 18, 2016 @ 18:33

            Hi Richards,

            Do you want to run your integration tests and end-to-end tests at the same time, or do you want to create different profiles for them (i.e. it is possible to run either integration or end-to-end tests)?

          • Richards Apr 20, 2016 @ 8:27

            Hi Petri,

            Sorry for the delay to respond to your question.
            I want to run integration tests and end-to-end tests separately. Yes, I would like to create different profiles for them.

            Thanks,
            Richards.

          • Petri Apr 21, 2016 @ 23:14

            Hi Richards,

            If I remember correctly, you can use a profile specific plugin configuration for this purpose. You can do this by following these steps:

            1. Configure the source directory of your end-to-end tests.
            2. Create a separate Maven profile for end-to-end tests.
            3. Create profile specific Failsafe plugin configurations for integration test and end-to-end test profiles. Remember to change the test class suffix (you can use E2ETest or something like that).

            If you don't know how to do this, let me know and I might write a blog post about this.

  • Stephane May 26, 2016 @ 17:18

    Is a test requiring an external database and no web layer considered an integration test ? Or does it need to involve a web layer to be considered an integration test ?

    • Petri May 27, 2016 @ 8:32

      Hi Stephane,

      I think that you just stirred a hornet's nest. There are a lot of people out there who cannot agree what these terms (unit and integration test) means. That being said, I think that the test you described is an integration test because it uses a database (either an external database or an in-memory database).

  • Brooke Smith Jul 14, 2016 @ 4:08

    Hi Petri,

    Great article - very helpful!

    The solution you presented prevents `-DskipTests` from skipping all tests. You need to do `-Dskip.unit.tests -Dskip.integration.tests`. Do you know of a way to get the former working? I played with this for a while and gave up.

    My solution was to not use the two switches. And I added an `**/IT*java` to the Unit Tests surefire and added a dummy `surefire` with `**/*java` to the integration-tests profile to prevent the unit tests running. Another difference is that my unit test surefire was outside of any profile - I'm not sure if this would make a difference in the case of your multi-profile configuration.

    Cheers,

    Brooke

    • Petri Jul 18, 2016 @ 23:34

      Hi Brooke,

      Ah. I have to admit that I have never thought of that because I haven't needed that "feature". I would probably just create a profile called: skipTests and configure it to skip both unit and integration tests (set the values of the skip.integration.tests and skip.unit.tests properties to true). This way I wouldn't have to make other changes to the configuration. However, I admit that I have no idea if this is the best way to skip all tests in this case.

  • Prasad Shitole Sep 13, 2016 @ 13:12

    Hey Petri,

    Can I configure Jacoco in Tomcat. I tried searching all the available information on google, but there is no data available which gives stepwise information to configure JaCoCo with Tomcat. Can you please help?

    • Petri Sep 19, 2016 @ 18:21

      Hi,

      It should be possible to do this (at least in theory). Unfortunately I have never done this myself. In other words, I don't know how you can do it.

  • Monis Oct 7, 2016 @ 8:46

    Hi Petri,

    Thanks for a nice blog, but in my case the dev and test code are in different repos, and i want to get code coverage of dev code by running Test Suite(Test Rest API services) to get my code coverage of dev code, is there any way i can instrument dev code first and then run test suite against it.

    Thanks,
    Monis.

    • Petri Oct 11, 2016 @ 20:01

      Hi,

      the JaCoCo agent must be running when the tests are run because this agent records that data that reveals which parts of the code were run by your tests.

      Also, if you want to deploy your code into an external server before you run your tests, you need to figure out a way to record this data on the server, copy it back to computer that runs the tests, and create the code coverage report. Unfortunately I don't know how you can create a build script that does this.

  • Howard Nov 21, 2016 @ 21:35

    This webpage is wonky.The characters are rendering as

    "&amp;lt;" (minus the quotations) for the

    All the other pages on your site show up fine.

    • Petri Nov 21, 2016 @ 22:51

      Oops. I fixed the problem. Thank you for pointing this out!

  • Vijay Yadav Sep 10, 2018 @ 22:14

    I have put my integration and unit test in the same folder like src/test/.../*IT.java.
    and I am executing the failsafe-plugin to run the integration test, but while displaying this in Jenkins as a post build action it is not able to publish in report, it is only giving the Unit test report not the integration test report.

    Please suggest.

  • Robert Apr 12, 2019 @ 18:46

    Hi Petri,
    Thank you for this blog - its great - I have followed it on my own work project but I am having an issue which I have log on SO now,
    https://stackoverflow.com/questions/55654489/how-to-stop-build-helper-maven-plugin-running-during-dev-goals-executions
    I am finding that when I run the Compile/Test goals for the DEV profile - the config.properties file for the Integration-Test directories is also being copied and over-writting the one from Dev-Test. I am not sure how to control this.

  • Anson Mar 10, 2022 @ 11:40

    Hi, Petri,
    per your suggestion , if project is a multi-module project, and we want to write integration tests for more than one module, it's better to create a separate integration test module for all tested module, is it right?

    • Petri Mar 11, 2022 @ 22:42

      Hi Anson,

      Thank you for your question. Sadly, I have to answer that it depends from your personal preferences, the requirements of your CI process, the used build tool, and the used testing framework.

      However, if I am writing tests with JUnit 5 and I use Maven, I tend to follow these two rules:

      • If I write integration tests which invoke code from multiple modules, I create a new module which contains these integration tests. This feels natural to me because my test code doesn't specify the behavior of a single module.
      • If I write integration tests which invoke code from one module, I add these tests to the module which contains the invoked code. I use this approach because I like to have my "executable specifications" in the same module as my production code.

      If you have any additional questions, don't hesitate to ask them.

Leave a Reply