Are you using inheritance in your tests? If so, check out three reasons why you should NOT do so.

Integration Testing with Maven

Share
Keyboard with oops key

Integration testing with Maven has always been a bit painful. Often the integration tests are scattered alongside with unit tests or a different module has been created for them. Both of these approaches are a bit disturbing.

Scattering integration tests in the same directory structure with unit tests is an awful idea because integration tests and unit tests are completely different creatures, and this approach forces us to mix them together. This a minor annoyance but this approach has a nasty side effect: running unit tests from IDE becomes a pain in the ass. When the tests are executed, our IDE wants to run all tests found from the test directory, which means that both unit tests and integration tests are executed. If we are “lucky”, this means that the tests take a bit longer to run, but often this means that the integration tests fails every time. Not nice, huh?

The second approach is a bit more feasible but to be honest, it feels like a total overkill. This forces us to turn our project in to a multi module project only because we want separate our integration from our unit tests. Also, if our project is already a multi module project and we want to write integration tests for more than one module, we are screwed. Of course we can always create a separate integration test module for each tested module, but it would be less painful to shoot ourselves to a leg instead.

This blog entry describes how we can separate the source directories of unit tests and integration tests and keep them in the same module. The requirements of our build process are following:

  • Integration and unit tests must have separate source folders.
  • Integration and unit tests must have separate configuration files.
  • Only unit tests are run by default.
  • It must be possible to run only integration tests.
  • Failing integration tests must cause build failure.

We can implement these requirements by following these steps:

  • Create a separate profile for development and integration testing.
  • Create profile specific configuration files.
  • Add a new source directory to our build.
  • Configure the Surefire Maven plugin.
  • Configure the Failsafe Maven plugin.

These steps are explained with more details in following.

Creating New Profiles for Development and Integration Testing

First we have to create two profiles: a profile that is used for development and a profile that is used for running the integration tests. We have two goals: First, we want to disable integration tests when development profile is used. Second, we want to disable unit tests when the integration test profile is used. In order to achieve these goals, we will introduce three new properties:

  • The skip.unit.tests property specifies whether unit tests are skipped or not.
  • The skip.integration.tests property specifies if integration tests are skipped or not.
  • The build.profile.id property identifies the used profile.

The configuration of the new build profiles has two steps:

  • Add the default values of the specified properties to our POM file.
  • Create the new profiles and extend the default property values in the integration-test profile.

The configuration of the Maven profiles is given in following:

<properties>
    <!-- Used to locate the profile specific configuration file. -->
    <build.profile.id>dev</build.profile.id>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <!-- Only unit tests are run by default. -->
    <skip.integration.tests>true</skip.integration.tests>
    <skip.unit.tests>false</skip.unit.tests>
</properties>
<profiles>
    <profile>
        <id>dev</id>
    </profile>
    <profile>
        <id>integration-test</id>
        <properties>
            <!-- Used to locate the profile specific configuration file. -->
            <build.profile.id>integration-test</build.profile.id>
            <!-- Only integration tests are run. -->
            <skip.integration.tests>false</skip.integration.tests>
            <skip.unit.tests>true</skip.unit.tests>
        </properties>
    </profile>
</profiles>

Creating Profile Specific Configuration Files

In order to create the profile specific configuration files, we have to use a concept called resource filtering. If you are not familiar with this concept, you might want to check out my blog entry, which explains how profile specific configuration files are created. The configuration of resource filtering has two steps:

  • Configure the location of the configuration file that contains profile specific configuration (The value of the build.profile.id property identifies the used profile).
  • Configure the location of the resource directory.

The required Maven configuration is given in following:

<filters>
    <filter>profiles/${build.profile.id}/config.properties</filter>
</filters>
<resources>
    <resource>
        <filtering>true</filtering>
        <directory>src/main/resources</directory>
    </resource>
</resources>

Adding New Source Directory for Integration Tests

Since Maven does not support multiple test source directories, we have to use the Build Helper Maven plugin. This plugin has a goal called add-test-source that is used to add a test source directory to a Maven build. In order to add the source directory of our integration tests to our Maven build, we have to follow these steps:

  • Ensure that the add-test-source goal of Builder Helper Maven plugin is executed at Maven’s generate-test-source lifecycle phase.
  • Configure the source directory of our integration tests.

The configuration of the Build Helper Maven plugin is given in following:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>1.7</version>
    <executions>
        <!-- States that the plugin's add-test-source goal is executed at generate-test-sources phase. -->
        <execution>
            <id>add-integration-test-sources</id>
            <phase>generate-test-sources</phase>
            <goals>
                <goal>add-test-source</goal>
            </goals>
            <configuration>
                <!-- Configures the source directory of integration tests. -->
                <sources>
                    <source>src/integration-test/java</source>
                </sources>
            </configuration>
        </execution>
    </executions>
</plugin>

Configuring the Surefire Maven Plugin

We will use the Surefire Maven plugin to run our unit tests. We can configure this plugin by following these steps:

  • Configure the plugin to skip unit tests if the value of the skip.unit.tests property is true.
  • Exclude our integration tests. We will assume that the name of each integration test class starts with a string “IT”.

The configuration of the Surefire Maven plugin is given in following:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12</version>
    <configuration>
        <!-- Skips unit tests if the value of skip.unit.tests property is true -->
        <skipTests>${skip.unit.tests}</skipTests>
        <!-- Excludes integration tests when unit tests are run. ß-->
        <excludes>
            <exclude>**/IT*.java</exclude>
        </excludes>
    </configuration>
</plugin>

Configuring the Failsafe Maven Plugin

The Failsafe Maven plugin is used to execute our integration tests. We can configure it by following these steps:

  • Configure the plugin to run its integration-test and verify goals.
  • Configure the plugin to skip integration tests if the value of the skip.integration.tests property is true.

The configuration of the Failsafe Maven plugin looks following:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.12</version>
    <executions>
        <!-- States that both integration-test and verify goals of the Failsafe Maven plugin are executed. -->
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
            <configuration>
                <!-- Skips integration tests if the value of skip.integration.tests property is true -->
                <skipTests>${skip.integration.tests}</skipTests>
            </configuration>
        </execution>
    </executions>
</plugin>

Running Unit and Integration Tests

That is all. We have now configured our pom.xml and its time to run our tests. The commands used to run both unit and integration tests are explained in following:

  • We can run our unit tests by executing a command mvn clean test on command line.
  • Our integration tests are executed by running a command mvn clean verify -P integration-test on command line. If we want to run our integration tests from our IDE, we have to manually mark the source directory of our integration tests as a test source root directory.

I have also created a very simple example project that contains one integration test and one unit test. This example project can be used to demonstrate the concept that is described in this blog entry. As always, the example project is available at Github.

If you enjoy reading similar content, you should follow me on Twitter:

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He specializes in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

20 comments… add one

  • If we are writing integration tests for real life applications, we often have to use integration test specific resource files (E.g. DBUnit dataset files). We can do this by ensuring that the add-test-resource goal of Builder Helper Maven plugin is executed at Maven’s generate-test-resources lifecycle phase.

    We do this by adding the following snippet to the executions section of the Builder Helper Maven plugin configuration:

    <execution>
    	<id>add-integration-test-resources</id>
    	<phase>generate-test-resources</phase>
    	<goals>
    		<goal>add-test-resource</goal>
    	</goals>
    	<configuration>
    		<resources>
    			<resource>
    				<directory>src/integration-test/resources</directory>
    			</resource>
    		</resources>
    	</configuration>
    </execution>
    

    This ensures that the directory src/integration-test/resources is added to our Maven build.

    Reply
  • Hey Petri,

    Very nice article.
    It was very helpful for me!!!
    Thanks a lot.

    Best Regards,
    Celso

    Reply
    • You are welcome! I am happy that I could help you out.

      Reply
  • Hi Petri,

    I want to use Jacoco for web application code coverage.
    I think the above only works fine if you are running tests and Web Server on the same machine.

    Currently my Jenkins run on one machine and my server is on another machines.Does it still work for that scenarior or what changes should I do for that.

    Reply
    • Hi Rang,

      If you are referring to this blog post, I have to confess that I have not tried running the example with Jenkins yet.

      I am planning to do this at some point because I want to make sure that it works. In the meantime, you might want to take a look at this blog post which describes the usage of the JaCoCo Jenkins plugin.

      Reply
    • Jacoco has a tcpmode option that can operate as either client or server. From the Jenkins CI – you can pull the coverage metrics and reset the collection. So as long as you start the server with the appropriate agent configuration you will be able to collect coverage data remotely (be advised ther is no security here – so one would only do this in a trusted environment)

      Reply
      • Thanks for the tip! I will try this out when I have got some time to do it and write a new blog post about it.

        Reply
  • Very helpful post. Thanks.

    Reply
    • You are welcome!

      Reply
  • It would sure be nice if we didn’t have the magic name requirement (**/IT*.java) for excluding integration tests from the unit test build. Do you know of a way to accomplish this?

    Reply
    • Yes, that would be nice.

      One option is to move the configuration of the Build Helper Maven plugin to the profile which you use to run the integration tests. This way the integration tests would be added to the classpath only when you use the correct profile which means that you can name your tests anyway you want.

      I haven’t tried this out but it should work (in theory).

      Reply
  • Hello, great contribution Petri.
    I used this configuration in a multi module project and it worked ok.
    The only thing is that using Eclipse (Kepler + Maven 3) I found that the eclipse plugin had a problem with the execution declaration in the maven helper plugin.
    If this happens to anyone else, here are the three possible solutions:
    1. In the quick fix (of eclipse) select the option of adding the ignore of the execution to the eclipse build in the eclipse project file. Even though it warns that it´s in an experimental fase, it works fine.
    2. In the quick fix (of eclipse) select the option of adding the ignore to the eclipse build in the project POM.
    3. Update the m2 lifecycle mapping connector. It connects to the marketplace and installs it. Restart eclipse and the problem is solved.

    These three approaches are independent.

    Hope this help to somebody.

    Reply
    • Hi Emiliano,

      It is nice to know that this tutorial was useful to you. Also, thank you for sharing the solutions for the problem you had with Eclipse and Build Helper Maven plugin. I am sure that this information is useful to someone!

      Reply
  • Hello Petri,
    Very informative and nice article.

    Need suggestion: I am planning to write integration test for my product.

    Product have 3 different web applications with 3 different spring configuration. Currently, Integration test is written to load just base configuration but I want to separate modules(applications) and write common integration test across modules ..so planing to create 4 modules like one for base and rest 3 like application specific modules so each specific module will load their own spring configuration while doing integration test.

    What do you think will be best approach? Your help and views will be greatly appreciated.
    Many Thanks

    Reply
    • Hi Tej,

      I would configure the application context in each web application module (no configuration in the core module). The reason for this is that if you have to override the base configuration, you can run into really weird problems. I have to admit that my approach means that you probably have to create similar application context configuration classes / files in each web application module but I am ready to make that tradeoff.

      Also, I would simply write integration tests for each web application. This way I could ensure that the base module is working correctly with each web application.

      I hope that this answered to your question. If you have any additional questions, feel free to ask them!

      Reply
  • This is by far one of the best posts I’ve read on Integration testing. The content is very well presented too.

    Keep up the great work.

    Reply
    • Thanks! I am happy to hear that this blog post was useful to you.

      Reply
  • Greetings,
    Very nice article! However, I am not quite being able to pull this off.
    “mvn clean verify -P integration-test” is correctly executing only the TestOneIT.java I have in my src/it/java folder

    However, the “mvn clean test ” is executing my jUnit tests inside src/test/java AND my TestOneIt.java as well :( Like the if the ${skip.integration.tests}
    inside the maven-failsafe-plugin isn’t doing nothing.

    Help?

    Reply
    • Sorry, my bad. the problem was the search pattern for the integration test files.
      It was just a matter of changing **/IT*.java to **/*IT*.java

      Reply
      • No worries. Good to hear that you were able to solve your problem!

        Reply

Leave a Comment