Are you looking for an advent calendar? You found it!

Integration Testing with Maven

Keyboard with oops key

Adding integration tests to a Maven build has traditionally been a bit painful. I suspect that the reason for this is that the standard directory layout has only one test directory (src/test).

If we want to use the standard directory layout and add integration tests to our Maven build, we have two options:

First, we can add our integration tests to the same directory as our unit tests. This is an awful idea because integration tests and unit tests are totally different beasts and this approach forces us to mix them. Also, if we follow this approach, running unit tests from our IDE becomes a pain in the ass. When we run tests, our IDE runs all tests found from the test directory. This means that both unit and integration tests are run. If we are “lucky”, this means that our test suite is slower than it could be, but often this means that our integration tests fail every time. Not nice, huh?

Second, we can add our integration tests to a new module. This is overkill because it forces us to transform our project into a multi-module project only because we want to separate our integration tests from our unit tests. Also, if our project is already a multi-module project, and we want to write integration tests for more than one module, we are screwed. Of course we can always create a separate integration test module for each tested module, but it would be less painful to shoot ourselves in the foot.

It is pretty clear that both of these solutions suck. This blog post describes how we can solve the problems of these solutions.

The requirements of our Maven build are:

  • Integration and unit tests must have separate source directories. The src/integration-test/java directory must contain the source code of our integration tests and the src/test/java directory must contain the source code of our unit tests.
  • Integration and unit tests must have different resource directories. The src/integration-test/resources directory must contain the resources of our integration tests and the src/test/resources directory must contain the resources of our unit tests.
  • Only unit tests are run by default.
  • It must be possible to run only integration tests.
  • If an integration test fails, it must fail our build.
  • The name of each integration test class must start with the prefix ‘IT’.

Let’s start by creating Maven profiles for unit and integration tests.

Creating Maven Profiles for Unit and Integration Tests

First, we need to create two Maven profiles that are described in the following:

The dev profile is used in the development environment, and it is the default profile of our Maven build (i.e. It is active when the active profile is not specified). This Maven profile has two goals that are described in the following:

  1. It configures the name of the directory that contains the properties file which contains the configuration used in the development environment.
  2. It ensures that only unit tests are run when the dev profile is active.

The integration-test profile is used for running the integration tests. The integration-test profile has two goals that are described in the following:

  1. It configures the name of the directory that contains the properties file which contains the configuration used by our integration tests.
  2. It ensures that only integration tests are run when the integration-test profile is active.

We can create these profiles by adding the following XML to our pom.xml file:

<profiles>
    <!-- The Configuration of the development profile -->
    <profile>
        <id>dev</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
        <properties>
            <!--
                Specifies the build.profile.id property that must be equal than the name of
                the directory that contains the profile specific configuration file.
                Because the name of the directory that contains the configuration file of the
                development profile is dev, we must set the value of the build.profile.id
                property to dev.
            -->
            <build.profile.id>dev</build.profile.id>
            <!--
                Only unit tests are run when the development profile is active
            -->
            <skip.integration.tests>true</skip.integration.tests>
            <skip.unit.tests>false</skip.unit.tests>
        </properties>
    </profile>
    <!-- The Configuration of the integration-test profile -->
    <profile>
        <id>integration-test</id>
        <properties>
            <!--
                Specifies the build.profile.id property that must be equal than the name of
                the directory that contains the profile specific configuration file.
                Because the name of the directory that contains the configuration file of the
                integration-test profile is integration-test, we must set the value of the
                build.profile.id property to integration-test.
            -->
            <build.profile.id>integration-test</build.profile.id>
            <!--
                Only integration tests are run when the integration-test profile is active
            -->
            <skip.integration.tests>false</skip.integration.tests>
            <skip.unit.tests>true</skip.unit.tests>
        </properties>
    </profile>
</profiles>

Second, we have to ensure that our application uses the correct configuration. When we run it by using the dev profile, we want that it uses the configuration that is used in the development environment. On the other hand, when we run our integration tests, we want that the tested code uses the configuration tailored for our integration tests.

We can ensure that our application uses the profile specific configuration by following these steps:

  1. Configure our Maven build to load the profile specific configuration file (config.properties) from the configuration directory of the active Maven profile.
  2. Configure the resource directory of our build (src/main/resources) and enable resources filtering. This ensures that the placeholders found from our resources are replaced with the actual property values that are read from the profile specific configuration file.

The relevant part of our POM file looks as follows:

<filters>
    <!--
        Ensures that the config.properties file is always loaded from the
        configuration directory of the active Maven profile.

    -->
    <filter>profiles/${build.profile.id}/config.properties</filter>
</filters>
<resources>
    <!--
        Placeholders that are found from the files located in the configured resource
        directories are replaced with the property values found from the profile
        specific configuration file.
    -->
    <resource>
        <filtering>true</filtering>
        <directory>src/main/resources</directory>
    </resource>
</resources>

Let’s move on and find out how we can add extra source and resource directories to our Maven build.

Adding Extra Source and Resource Directories to Our Maven Build

The requirements of our Maven build state that our integration and unit tests must have separate source and resource directories. Because the source code and the resources of our unit tests are found from the standard test directories, we have to create new source and resource directories for our integration tests.

We can add extra source and resource directories to our build by using the Build Helper Maven Plugin. We fulfill our requirements by following these steps:

  1. Add the Build Helper Maven Plugin to the plugins section of our POM file.
  2. Configure the executions that add the new source and resource directories to our build.

First, we can add the Build Helper Maven Plugin to our Maven build by adding the following XML to the plugins section of our pom.xml file:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>1.9.1</version>
    <executions>
        <!-- Add executions here -->
    </executions>
</plugin>

Second, we have to configure the executions that add our new source and resource directories to our Maven build. We can do this by following these steps:

  1. Create an execution that adds the new source directory (src/integration-test/java) to our build.
  2. Create an execution that adds the new resource directory (src/integration-test/resources) to our build and enable resource filtering.

We can create these executions by adding the following XML to the configuration of the Build Helper Maven Plugin:

<!-- Add a new source directory to our build -->
<execution>
    <id>add-integration-test-sources</id>
    <phase>generate-test-sources</phase>
    <goals>
        <goal>add-test-source</goal>
    </goals>
    <configuration>
        <!-- Configures the source directory of our integration tests -->
        <sources>
            <source>src/integration-test/java</source>
        </sources>
    </configuration>
</execution>
        <!-- Add a new resource directory to our build -->
<execution>
<id>add-integration-test-resources</id>
<phase>generate-test-resources</phase>
<goals>
    <goal>add-test-resource</goal>
</goals>
<configuration>
    <!-- Configures the resource directory of our integration tests -->
    <resources>
        <!--
            Placeholders that are found from the files located in the configured resource
            directories are replaced with the property values found from the profile
            specific configuration file.
        -->
        <resource>
            <filtering>true</filtering>
            <directory>src/integration-test/resources</directory>
        </resource>
    </resources>
</configuration>
</execution>

Let’s find out how we can run our unit tests with Maven.

Running Unit Tests

If we think about the requirements of our Maven build, we notice that we must take the following things into account when we run our unit tests:

  • Unit tests are run only when the dev profile is active.
  • When we run our unit tests, our build must run all test methods found from the test classes whose name don’t start with the prefix ‘IT’.

We can run our unit tests by using the Maven Surefire plugin. We can configure it by following this steps:

  1. Configure the plugin to skip unit tests if the value of the skip.unit.tests property is true.
  2. Configure the plugin to ignore our integration tests.

We can configure the Maven Surefire plugin by adding the following XML to the plugins section of our POM file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18</version>
    <configuration>
        <!-- Skips unit tests if the value of skip.unit.tests property is true -->
        <skipTests>${skip.unit.tests}</skipTests>
        <!-- Excludes integration tests when unit tests are run -->
        <excludes>
            <exclude>**/IT*.java</exclude>
        </excludes>
    </configuration>
</plugin>

We can run our unit tests by using the command: mvn clean test -P dev.

Let’s move on and find out how we can run our integration tests with Maven.

Running Integration Tests

If we think about the requirements of our Maven build, we notice that we have to take the following things into account when we run our integration tests:

  • Integration tests are run only when the integration-test profile is active.
  • If an integration test fails, our build must fail as well.
  • When we run our integration tests, our build must run all test methods found from the test classes whose name starts with the prefix ‘IT’.

We can run our integration tests by using the Maven Failsafe plugin. We can configure it by following these steps:

  1. Create an execution that invokes the integration-test and verify goals of the Maven Failsafe plugin when we run our integration tests. The integration-test goal runs our integration tests in the integration-test phase of the Maven default lifecycle. The verify goal verifies the results of our integration test suite and fails the build if it finds failed integration tests.
  2. Configure the execution to skip integration tests if the value of the skip.integration.tests property is true.

We can configure the Maven Failsafe Plugin by adding the following XML to the plugins section of our POM file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.18</version>
    <executions>
        <!--
            Invokes both the integration-test and the verify goals of the
            Failsafe Maven plugin
        -->
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
            <configuration>
                <!--
                    Skips integration tests if the value of skip.integration.tests
                    property is true
                -->
                <skipTests>${skip.integration.tests}</skipTests>
            </configuration>
        </execution>
    </executions>
</plugin>

We can run our integration tests by using the command: mvn clean verify -P integration-test.

Let’s move on and summarize what we learned from this blog post.

Summary

This blog post has taught us three things:

  • We can add extra source or resource directories to our Maven build by using the Build Helper Maven Plugin.
  • We can run our unit tests by using the Maven Surefire plugin.
  • We can run our integration tests by using the Maven Failsafe plugin.
If you enjoyed reading this blog post, you should follow me on Twitter:

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He is specialized in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

43 comments… add one

  • If we are writing integration tests for real life applications, we often have to use integration test specific resource files (E.g. DBUnit dataset files). We can do this by ensuring that the add-test-resource goal of Builder Helper Maven plugin is executed at Maven’s generate-test-resources lifecycle phase.

    We do this by adding the following snippet to the executions section of the Builder Helper Maven plugin configuration:

    <execution>
    	<id>add-integration-test-resources</id>
    	<phase>generate-test-resources</phase>
    	<goals>
    		<goal>add-test-resource</goal>
    	</goals>
    	<configuration>
    		<resources>
    			<resource>
    				<directory>src/integration-test/resources</directory>
    			</resource>
    		</resources>
    	</configuration>
    </execution>
    

    This ensures that the directory src/integration-test/resources is added to our Maven build.

    Reply
  • Hey Petri,

    Very nice article.
    It was very helpful for me!!!
    Thanks a lot.

    Best Regards,
    Celso

    Reply
    • You are welcome! I am happy that I could help you out.

      Reply
  • Hi Petri,

    I want to use Jacoco for web application code coverage.
    I think the above only works fine if you are running tests and Web Server on the same machine.

    Currently my Jenkins run on one machine and my server is on another machines.Does it still work for that scenarior or what changes should I do for that.

    Reply
    • Hi Rang,

      If you are referring to this blog post, I have to confess that I have not tried running the example with Jenkins yet.

      I am planning to do this at some point because I want to make sure that it works. In the meantime, you might want to take a look at this blog post which describes the usage of the JaCoCo Jenkins plugin.

      Reply
    • Jacoco has a tcpmode option that can operate as either client or server. From the Jenkins CI – you can pull the coverage metrics and reset the collection. So as long as you start the server with the appropriate agent configuration you will be able to collect coverage data remotely (be advised ther is no security here – so one would only do this in a trusted environment)

      Reply
      • Thanks for the tip! I will try this out when I have got some time to do it and write a new blog post about it.

        Reply
  • Very helpful post. Thanks.

    Reply
    • You are welcome!

      Reply
  • It would sure be nice if we didn’t have the magic name requirement (**/IT*.java) for excluding integration tests from the unit test build. Do you know of a way to accomplish this?

    Reply
    • Yes, that would be nice.

      One option is to move the configuration of the Build Helper Maven plugin to the profile which you use to run the integration tests. This way the integration tests would be added to the classpath only when you use the correct profile which means that you can name your tests anyway you want.

      I haven’t tried this out but it should work (in theory).

      Reply
  • Hello, great contribution Petri.
    I used this configuration in a multi module project and it worked ok.
    The only thing is that using Eclipse (Kepler + Maven 3) I found that the eclipse plugin had a problem with the execution declaration in the maven helper plugin.
    If this happens to anyone else, here are the three possible solutions:
    1. In the quick fix (of eclipse) select the option of adding the ignore of the execution to the eclipse build in the eclipse project file. Even though it warns that it´s in an experimental fase, it works fine.
    2. In the quick fix (of eclipse) select the option of adding the ignore to the eclipse build in the project POM.
    3. Update the m2 lifecycle mapping connector. It connects to the marketplace and installs it. Restart eclipse and the problem is solved.

    These three approaches are independent.

    Hope this help to somebody.

    Reply
    • Hi Emiliano,

      It is nice to know that this tutorial was useful to you. Also, thank you for sharing the solutions for the problem you had with Eclipse and Build Helper Maven plugin. I am sure that this information is useful to someone!

      Reply
    • Approaches 1 & 2 deprive you of a properly setup .classpath (i.e. src/integration-test/java appears as a source folder in Eclipse).

      BTW For the latest version of ‘m2e connector for build-helper-maven-plugin’ see this thread http://comments.gmane.org/gmane.comp.ide.eclipse.plugins.m2eclipse.user/10082

      Reply
      • Thank you for leaving this comment! I am sure that it is useful for Eclipse users.

        Reply
  • Hello Petri,
    Very informative and nice article.

    Need suggestion: I am planning to write integration test for my product.

    Product have 3 different web applications with 3 different spring configuration. Currently, Integration test is written to load just base configuration but I want to separate modules(applications) and write common integration test across modules ..so planing to create 4 modules like one for base and rest 3 like application specific modules so each specific module will load their own spring configuration while doing integration test.

    What do you think will be best approach? Your help and views will be greatly appreciated.
    Many Thanks

    Reply
    • Hi Tej,

      I would configure the application context in each web application module (no configuration in the core module). The reason for this is that if you have to override the base configuration, you can run into really weird problems. I have to admit that my approach means that you probably have to create similar application context configuration classes / files in each web application module but I am ready to make that tradeoff.

      Also, I would simply write integration tests for each web application. This way I could ensure that the base module is working correctly with each web application.

      I hope that this answered to your question. If you have any additional questions, feel free to ask them!

      Reply
  • This is by far one of the best posts I’ve read on Integration testing. The content is very well presented too.

    Keep up the great work.

    Reply
    • Thanks! I am happy to hear that this blog post was useful to you.

      Reply
  • Greetings,
    Very nice article! However, I am not quite being able to pull this off.
    “mvn clean verify -P integration-test” is correctly executing only the TestOneIT.java I have in my src/it/java folder

    However, the “mvn clean test ” is executing my jUnit tests inside src/test/java AND my TestOneIt.java as well :( Like the if the ${skip.integration.tests}
    inside the maven-failsafe-plugin isn’t doing nothing.

    Help?

    Reply
    • Sorry, my bad. the problem was the search pattern for the integration test files.
      It was just a matter of changing **/IT*.java to **/*IT*.java

      Reply
      • No worries. Good to hear that you were able to solve your problem!

        Reply
  • good one :)

    Reply
    • Thanks! I am happy to hear that you liked this blog post.

      Reply
  • Hi Petri, I’m working in a multi-module setting, I have:

    pom.xml
    – persistence
    – app-services
    – rest

    Each of these modules have both unit and integration tests, and I was wondering if I can just declare all of these profile settings in the parent pom? The reason I’m not sure is because of the filtering, the parent won’t have any resources to filter (it’s just a “pom”)

    I know this is a candidate for TIAS, but I figured if you knew off the top of your head it would save me time mocking up a bunch of tests.

    Reply
  • Hi Petri and thank you for your very helpful tutorial!

    I’m trying to implement integration-testing in the way you describe. In my situation I also need to keep test .xml context configuration files separated by ones of application. So I put them into /src/integration-test/resources/META-INF.

    Looking in target folder I have found that: files in src/main/resources are placed under PROJECT/target/classes/, while files in /src/integration-test/resources/ are placed in PROJECT/target/test-classes

    application.properties in PROJECT/target/classes is processed by maven plugins that replace ${placeholder} with value of ${profile}/config.properties, while application.properties in PROJECT/target/test-classes is not processed. So executing maven integration-test I get exception like: Circular placeholder reference ‘db.driver’ in property definitions.

    Trying to put application.properties without placeholders in src/integration-test/resources seems than application.properties is not loaded by @PropertySource(“classpath:application.properties”). I get exception like: Access to DialectResolutionInfo cannot be null when ‘hibernate.dialect’ not set.

    So I’m wrong in setting my test context, or I need to modify my pom to reach my requirements?

    Another question is, what if I want to execute/debug each single test from eclipse?

    Thank you very much Petri!

    Have a good day

    Reply
    • Hi Marco,

      The idea behind the configuration explained here is that you can use the same properties file (src/main/resources/application.properties) in your application and in your tests because Maven selects the actual property values based on the active profile.

      In other words,

      • If the active profile is dev the placeholders found from the src/main/resources/application.properties file are replaced with the property values found from the profiles/dev/config.properties file.
      • If the active profile is integration-test the placeholders found from the src/main/resources/application.properties file are replaced with the property values found from the profiles/integration-test/config.properties file.

      This means that you don’t need to create a separate properties file to the src/integration-test/resources directory because you can use the properties file found from the src/main/resources directory. This file contains the test specific configuration as long as the correct profile (integration-test) is active.

      You can the refer to this file in your test specific application context configuration file like this: ‘classpath:application.properties’.

      On the other hand, if you want to create a separate properties file for your integration tests, you shouldn’t name it application.properties because if you do, the classpath has two files called application.properties and this can cause problems if you refer to them by using the notation: ‘classpath:application.properties’ (the classloader loads one of them but you cannot know which one).

      I hope that this shed some light to your properties file related problems.

      I haven’t used Eclipse for six years but if I have to run / debug a single integration test in my IDE (IntelliJ Idea), I follow these steps:

      1. Compile the project when the integration-test profile is active. This is required so that the correct property values are filtered to the application.properties file.
      2. Run the test in IDE. Typically I can just click the name of the test method with the right mouse button and select ‘Run test’. I assume that Eclipse is working in a similar way.

      I hope that this answered to your questions. If not, let me know!

      Reply
      • Thank you Petri for your reply!

        I have tryed to use /src/main/resources/application.properties in conjunction with .xml configuration files in /src/intergation-test/resources/META-INF/*-test.xml (I have renamed them with test suffix) but context initialization fails: placeholders cannot be found.

        Here is how test is configured:

        
        @RunWith(SpringJUnit4ClassRunner.class)
        @PropertySource("classpath:application.properties")
        @ContextConfiguration(locations={
        		"classpath:/META-INF/datasource_test.xml",
        		"classpath:/META-INF/jpa-tx-config_test.xml"
        		})
        @TestExecutionListeners({ DependencyInjectionTestExecutionListener.class,
        	DirtiesContextTestExecutionListener.class,
        	TransactionalTestExecutionListener.class,
        	DbUnitTestExecutionListener.class })
        @DirtiesContext(classMode=ClassMode.AFTER_EACH_TEST_METHOD)
        @ActiveProfiles("dev")
        
        

        So application.properties can be found on target/classes, with other src/main/resources configuration files, while test configuration files can be found on target/test-classes.

        Seems that if application.properties is not in same folder of xmls, placeholders cannot be resolved..

        Reply
        • Hi Marco,

          I noticed that you use the @PropertySource annotation in your test class. I assume that you are trying to load the properties file which contains the property values used in your context configuration files. Am I correct?

          The thing is that the javadoc of the @PropertySource annotation states that it works only if it is used together with the @Configuration annotation. In other words, you should use Java configuration.

          If you want to use XML configuration, you can configure the used properties file by using the property-placeholder tag of the context namespace.

          You could also verify that the property values are found from the application.properties file after it has been copied to the target/classes directory. If this is the case, you know that the problem isn’t in your build script. If the values are not found from this file, the problem is that Maven isn’t filtering the property values from your profile specific configuration files to the application.properties file.

          By the way, if you cannot solve this problem, it would be helpful to see your build script / project. Is this possible or is it a work related project?

          Reply
          • Thank you Petri, you give me the right hint.

            Just enable in xml do the trick! So I hare removed @PropertySource annotation (that as documentation states works only if it is used together with the @Configuration annotation).

            It also works using xml files in src/integration-testing/resources, preferred over that ones defined in src/main/resources

            Thank you very much Petri!! Have a nice day

          • It also works using xml files in src/integration-testing/resources, preferred over that ones defined in src/main/resources: not true because when I run application from my eclipse workspace xml files conflict.. but I have solved appending a suffix to each xml file.

          • Hi,

            It is good to hear that you were able to solve your problem.

            I have always used the same XML configuration files for every profile (dev, test, prod, and integration-test), and moved the configuration to profile specific configuration files (although nowadays I use only Java configuration).

            On the other hand, this might not always be good enough because you might want to use different beans when you run your integration tests. I have solved this problem by using bean definition profiles (aka Spring profiles).

  • Hi Petri,
    This solved the problem I was having, very good post, thanks a lot.
    Tip: add a bonus to this entry and show how to run tests in parellel, it’s very handy and explained here http://maven.apache.org/surefire/maven-failsafe-plugin/examples/junit.html.

    Cheers
    Tom

    Reply
    • Hi Tom,

      I am happy to hear that this blog post was useful. Also, I will update this blog post when I have got time to do it and include the tip you mentioned in it. Thanks for sharing.

      Reply
  • Thanks a loooooot!!!
    you made my day!!!

    Reply
    • You are welcome! I am happy to hear that this blog post was useful to you.

      Reply
  • Dear Petri,
    I’m currently trying to generate code coverage reports Java Server for manual tests in a Maven project. I’m using eclipse with the JaCoCO plugin.

    The reports are properly generated when using automated Junit tests.

    So What I need is

    1. Run the program
    2. Perform some function on UI(Manual Testing)
    3. Exit the program
    4. Get a code coverage report for that run

    So can suggest me some way to achieve it.
    Thanks in advance.

    Best Regards,
    Krishna Singh

    Reply
  • Thanks a lot. It’s very helpful for me!

    Reply
    • You are welcome! I am happy to hear that this blog post was useful to you.

      Reply
  • Hi,
    Thank you for this great article. I am completely new to this world but voila! i was able to work with the steps mentioned above. However I am facing 1 issue.
    maven while building is looking for config.properties under {basedir}\profiles\* than {basedir}\src\main\profiles.
    Any clue?

    Reply
    • If you want to change the location of the profiles directory, you have to change the value of the filter element.

      For example, if you want to put the profile specific configuration directories to the src/main/profiles directory, you have to use the following configuration:

      
      <filters>
          <filter>src/main/profiles/${build.profile.id}/config.properties</filter>
      </filters>
      
      

      If you want to get more information about this, you should read this blog post.

      Reply

Leave a Comment