Do want to get a better understanding of Spring web application architecture? If so, get started right now!

Integration Testing with Maven

Keyboard with oops key

Integration testing with Maven has always been a bit painful. Often the integration tests are scattered alongside with unit tests or a different module has been created for them. Both of these approaches are a bit disturbing.

Scattering integration tests in the same directory structure with unit tests is an awful idea because integration tests and unit tests are completely different creatures, and this approach forces us to mix them together. This a minor annoyance but this approach has a nasty side effect: running unit tests from IDE becomes a pain in the ass. When the tests are executed, our IDE wants to run all tests found from the test directory, which means that both unit tests and integration tests are executed. If we are “lucky”, this means that the tests take a bit longer to run, but often this means that the integration tests fails every time. Not nice, huh?

The second approach is a bit more feasible but to be honest, it feels like a total overkill. This forces us to turn our project in to a multi module project only because we want separate our integration from our unit tests. Also, if our project is already a multi module project and we want to write integration tests for more than one module, we are screwed. Of course we can always create a separate integration test module for each tested module, but it would be less painful to shoot ourselves to a leg instead.

This blog entry describes how we can separate the source directories of unit tests and integration tests and keep them in the same module. The requirements of our build process are following:

  • Integration and unit tests must have separate source folders.
  • Integration and unit tests must have separate configuration files.
  • Only unit tests are run by default.
  • It must be possible to run only integration tests.
  • Failing integration tests must cause build failure.

We can implement these requirements by following these steps:

  • Create a separate profile for development and integration testing.
  • Create profile specific configuration files.
  • Add a new source directory to our build.
  • Configure the Surefire Maven plugin.
  • Configure the Failsafe Maven plugin.

These steps are explained with more details in following.

Creating New Profiles for Development and Integration Testing

First we have to create two profiles: a profile that is used for development and a profile that is used for running the integration tests. We have two goals: First, we want to disable integration tests when development profile is used. Second, we want to disable unit tests when the integration test profile is used. In order to achieve these goals, we will introduce three new properties:

  • The skip.unit.tests property specifies whether unit tests are skipped or not.
  • The skip.integration.tests property specifies if integration tests are skipped or not.
  • The build.profile.id property identifies the used profile.

The configuration of the new build profiles has two steps:

  • Add the default values of the specified properties to our POM file.
  • Create the new profiles and extend the default property values in the integration-test profile.

The configuration of the Maven profiles is given in following:

<properties>
	<!-- Used to locate the profile specific configuration file. -->
	<build.profile.id>dev</build.profile.id>
	<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	<!-- Only unit tests are run by default. -->
	<skip.integration.tests>true</skip.integration.tests>
	<skip.unit.tests>false</skip.unit.tests>
</properties>
<profiles>
	<profile>
    	<id>dev</id>
	</profile>
	<profile>
		<id>integration-test</id>
		<properties>
			<!-- Used to locate the profile specific configuration file. -->
			<build.profile.id>integration-test</build.profile.id>
			<!-- Only integration tests are run. -->
			<skip.integration.tests>false</skip.integration.tests>
			<skip.unit.tests>true</skip.unit.tests>
		</properties>
	</profile>
</profiles>

Creating Profile Specific Configuration Files

In order to create the profile specific configuration files, we have to use a concept called resource filtering. If you are not familiar with this concept, you might want to check out my blog entry, which explains how profile specific configuration files are created. The configuration of resource filtering has two steps:

  • Configure the location of the configuration file that contains profile specific configuration (The value of the build.profile.id property identifies the used profile).
  • Configure the location of the resource directory.

The required Maven configuration is given in following:

<filters>
	<filter>profiles/${build.profile.id}/config.properties</filter>
</filters>
<resources>
	<resource>
		<filtering>true</filtering>
		<directory>src/main/resources</directory>
	</resource>
</resources>

Adding New Source Directory for Integration Tests

Since Maven does not support multiple test source directories, we have to use the Build Helper Maven plugin. This plugin has a goal called add-test-source that is used to add a test source directory to a Maven build. In order to add the source directory of our integration tests to our Maven build, we have to follow these steps:

  • Ensure that the add-test-source goal of Builder Helper Maven plugin is executed at Maven’s generate-test-source lifecycle phase.
  • Configure the source directory of our integration tests.

The configuration of the Build Helper Maven plugin is given in following:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>1.7</version>
    <executions>
        <!-- States that the plugin's add-test-source goal is executed at generate-test-sources phase. -->
        <execution>
            <id>add-integration-test-sources</id>
            <phase>generate-test-sources</phase>
            <goals>
                <goal>add-test-source</goal>
            </goals>
            <configuration>
                <!-- Configures the source directory of integration tests. -->
                <sources>
                    <source>src/integration-test/java</source>
                </sources>
            </configuration>
        </execution>
    </executions>
</plugin>

Configuring the Surefire Maven Plugin

We will use the Surefire Maven plugin to run our unit tests. We can configure this plugin by following these steps:

  • Configure the plugin to skip unit tests if the value of the skip.unit.tests property is true.
  • Exclude our integration tests. We will assume that the name of each integration test class starts with a string “IT”.

The configuration of the Surefire Maven plugin is given in following:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-surefire-plugin</artifactId>
	<version>2.12</version>
	<configuration>
		<!-- Skips unit tests if the value of skip.unit.tests property is true -->
		<skipTests>${skip.unit.tests}</skipTests>
		<!-- Excludes integration tests when unit tests are run. ß-->
		<excludes>
			<exclude>**/IT*.java</exclude>
		</excludes>
	</configuration>
</plugin>

Configuring the Failsafe Maven Plugin

The Failsafe Maven plugin is used to execute our integration tests. We can configure it by following these steps:

  • Configure the plugin to run its integration-test and verify goals.
  • Configure the plugin to skip integration tests if the value of the skip.integration.tests property is true.

The configuration of the Failsafe Maven plugin looks following:

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-failsafe-plugin</artifactId>
	<version>2.12</version>
	<executions>
		<!-- States that both integration-test and verify goals of the Failsafe Maven plugin are executed. -->
		<execution>
			<id>integration-tests</id>
			<goals>
				<goal>integration-test</goal>
				<goal>verify</goal>
			</goals>
			<configuration>
				<!-- Skips integration tests if the value of skip.integration.tests property is true -->
				<skipTests>${skip.integration.tests}</skipTests>
			</configuration>
		</execution>
	</executions>
</plugin>

Running Unit and Integration Tests

That is all. We have now configured our pom.xml and its time to run our tests. The commands used to run both unit and integration tests are explained in following:

  • We can run our unit tests by executing a command mvn clean test on command line.
  • Our integration tests are executed by running a command mvn clean verify -P integration-test on command line. If we want to run our integration tests from our IDE, we have to manually mark the source directory of our integration tests as a test source root directory.

I have also created a very simple example project that contains one integration test and one unit test. This example project can be used to demonstrate the concept that is described in this blog entry. As always, the example project is available at Github.

If you enjoyed reading this blog post, you should follow me on Twitter:

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He is specialized in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

39 comments… add one

  • If we are writing integration tests for real life applications, we often have to use integration test specific resource files (E.g. DBUnit dataset files). We can do this by ensuring that the add-test-resource goal of Builder Helper Maven plugin is executed at Maven’s generate-test-resources lifecycle phase.

    We do this by adding the following snippet to the executions section of the Builder Helper Maven plugin configuration:

    <execution>
    	<id>add-integration-test-resources</id>
    	<phase>generate-test-resources</phase>
    	<goals>
    		<goal>add-test-resource</goal>
    	</goals>
    	<configuration>
    		<resources>
    			<resource>
    				<directory>src/integration-test/resources</directory>
    			</resource>
    		</resources>
    	</configuration>
    </execution>
    

    This ensures that the directory src/integration-test/resources is added to our Maven build.

    Reply
  • Hey Petri,

    Very nice article.
    It was very helpful for me!!!
    Thanks a lot.

    Best Regards,
    Celso

    Reply
    • You are welcome! I am happy that I could help you out.

      Reply
  • Hi Petri,

    I want to use Jacoco for web application code coverage.
    I think the above only works fine if you are running tests and Web Server on the same machine.

    Currently my Jenkins run on one machine and my server is on another machines.Does it still work for that scenarior or what changes should I do for that.

    Reply
    • Hi Rang,

      If you are referring to this blog post, I have to confess that I have not tried running the example with Jenkins yet.

      I am planning to do this at some point because I want to make sure that it works. In the meantime, you might want to take a look at this blog post which describes the usage of the JaCoCo Jenkins plugin.

      Reply
    • Jacoco has a tcpmode option that can operate as either client or server. From the Jenkins CI – you can pull the coverage metrics and reset the collection. So as long as you start the server with the appropriate agent configuration you will be able to collect coverage data remotely (be advised ther is no security here – so one would only do this in a trusted environment)

      Reply
      • Thanks for the tip! I will try this out when I have got some time to do it and write a new blog post about it.

        Reply
  • Very helpful post. Thanks.

    Reply
    • You are welcome!

      Reply
  • It would sure be nice if we didn’t have the magic name requirement (**/IT*.java) for excluding integration tests from the unit test build. Do you know of a way to accomplish this?

    Reply
    • Yes, that would be nice.

      One option is to move the configuration of the Build Helper Maven plugin to the profile which you use to run the integration tests. This way the integration tests would be added to the classpath only when you use the correct profile which means that you can name your tests anyway you want.

      I haven’t tried this out but it should work (in theory).

      Reply
  • Hello, great contribution Petri.
    I used this configuration in a multi module project and it worked ok.
    The only thing is that using Eclipse (Kepler + Maven 3) I found that the eclipse plugin had a problem with the execution declaration in the maven helper plugin.
    If this happens to anyone else, here are the three possible solutions:
    1. In the quick fix (of eclipse) select the option of adding the ignore of the execution to the eclipse build in the eclipse project file. Even though it warns that it´s in an experimental fase, it works fine.
    2. In the quick fix (of eclipse) select the option of adding the ignore to the eclipse build in the project POM.
    3. Update the m2 lifecycle mapping connector. It connects to the marketplace and installs it. Restart eclipse and the problem is solved.

    These three approaches are independent.

    Hope this help to somebody.

    Reply
    • Hi Emiliano,

      It is nice to know that this tutorial was useful to you. Also, thank you for sharing the solutions for the problem you had with Eclipse and Build Helper Maven plugin. I am sure that this information is useful to someone!

      Reply
    • Approaches 1 & 2 deprive you of a properly setup .classpath (i.e. src/integration-test/java appears as a source folder in Eclipse).

      BTW For the latest version of ‘m2e connector for build-helper-maven-plugin’ see this thread http://comments.gmane.org/gmane.comp.ide.eclipse.plugins.m2eclipse.user/10082

      Reply
      • Thank you for leaving this comment! I am sure that it is useful for Eclipse users.

        Reply
  • Hello Petri,
    Very informative and nice article.

    Need suggestion: I am planning to write integration test for my product.

    Product have 3 different web applications with 3 different spring configuration. Currently, Integration test is written to load just base configuration but I want to separate modules(applications) and write common integration test across modules ..so planing to create 4 modules like one for base and rest 3 like application specific modules so each specific module will load their own spring configuration while doing integration test.

    What do you think will be best approach? Your help and views will be greatly appreciated.
    Many Thanks

    Reply
    • Hi Tej,

      I would configure the application context in each web application module (no configuration in the core module). The reason for this is that if you have to override the base configuration, you can run into really weird problems. I have to admit that my approach means that you probably have to create similar application context configuration classes / files in each web application module but I am ready to make that tradeoff.

      Also, I would simply write integration tests for each web application. This way I could ensure that the base module is working correctly with each web application.

      I hope that this answered to your question. If you have any additional questions, feel free to ask them!

      Reply
  • This is by far one of the best posts I’ve read on Integration testing. The content is very well presented too.

    Keep up the great work.

    Reply
    • Thanks! I am happy to hear that this blog post was useful to you.

      Reply
  • Greetings,
    Very nice article! However, I am not quite being able to pull this off.
    “mvn clean verify -P integration-test” is correctly executing only the TestOneIT.java I have in my src/it/java folder

    However, the “mvn clean test ” is executing my jUnit tests inside src/test/java AND my TestOneIt.java as well :( Like the if the ${skip.integration.tests}
    inside the maven-failsafe-plugin isn’t doing nothing.

    Help?

    Reply
    • Sorry, my bad. the problem was the search pattern for the integration test files.
      It was just a matter of changing **/IT*.java to **/*IT*.java

      Reply
      • No worries. Good to hear that you were able to solve your problem!

        Reply
  • good one :)

    Reply
    • Thanks! I am happy to hear that you liked this blog post.

      Reply
  • Hi Petri, I’m working in a multi-module setting, I have:

    pom.xml
    – persistence
    – app-services
    – rest

    Each of these modules have both unit and integration tests, and I was wondering if I can just declare all of these profile settings in the parent pom? The reason I’m not sure is because of the filtering, the parent won’t have any resources to filter (it’s just a “pom”)

    I know this is a candidate for TIAS, but I figured if you knew off the top of your head it would save me time mocking up a bunch of tests.

    Reply
  • Hi Petri and thank you for your very helpful tutorial!

    I’m trying to implement integration-testing in the way you describe. In my situation I also need to keep test .xml context configuration files separated by ones of application. So I put them into /src/integration-test/resources/META-INF.

    Looking in target folder I have found that: files in src/main/resources are placed under PROJECT/target/classes/, while files in /src/integration-test/resources/ are placed in PROJECT/target/test-classes

    application.properties in PROJECT/target/classes is processed by maven plugins that replace ${placeholder} with value of ${profile}/config.properties, while application.properties in PROJECT/target/test-classes is not processed. So executing maven integration-test I get exception like: Circular placeholder reference ‘db.driver’ in property definitions.

    Trying to put application.properties without placeholders in src/integration-test/resources seems than application.properties is not loaded by @PropertySource(“classpath:application.properties”). I get exception like: Access to DialectResolutionInfo cannot be null when ‘hibernate.dialect’ not set.

    So I’m wrong in setting my test context, or I need to modify my pom to reach my requirements?

    Another question is, what if I want to execute/debug each single test from eclipse?

    Thank you very much Petri!

    Have a good day

    Reply
    • Hi Marco,

      The idea behind the configuration explained here is that you can use the same properties file (src/main/resources/application.properties) in your application and in your tests because Maven selects the actual property values based on the active profile.

      In other words,

      • If the active profile is dev the placeholders found from the src/main/resources/application.properties file are replaced with the property values found from the profiles/dev/config.properties file.
      • If the active profile is integration-test the placeholders found from the src/main/resources/application.properties file are replaced with the property values found from the profiles/integration-test/config.properties file.

      This means that you don’t need to create a separate properties file to the src/integration-test/resources directory because you can use the properties file found from the src/main/resources directory. This file contains the test specific configuration as long as the correct profile (integration-test) is active.

      You can the refer to this file in your test specific application context configuration file like this: ‘classpath:application.properties’.

      On the other hand, if you want to create a separate properties file for your integration tests, you shouldn’t name it application.properties because if you do, the classpath has two files called application.properties and this can cause problems if you refer to them by using the notation: ‘classpath:application.properties’ (the classloader loads one of them but you cannot know which one).

      I hope that this shed some light to your properties file related problems.

      I haven’t used Eclipse for six years but if I have to run / debug a single integration test in my IDE (IntelliJ Idea), I follow these steps:

      1. Compile the project when the integration-test profile is active. This is required so that the correct property values are filtered to the application.properties file.
      2. Run the test in IDE. Typically I can just click the name of the test method with the right mouse button and select ‘Run test’. I assume that Eclipse is working in a similar way.

      I hope that this answered to your questions. If not, let me know!

      Reply
      • Thank you Petri for your reply!

        I have tryed to use /src/main/resources/application.properties in conjunction with .xml configuration files in /src/intergation-test/resources/META-INF/*-test.xml (I have renamed them with test suffix) but context initialization fails: placeholders cannot be found.

        Here is how test is configured:

        
        @RunWith(SpringJUnit4ClassRunner.class)
        @PropertySource("classpath:application.properties")
        @ContextConfiguration(locations={
        		"classpath:/META-INF/datasource_test.xml",
        		"classpath:/META-INF/jpa-tx-config_test.xml"
        		})
        @TestExecutionListeners({ DependencyInjectionTestExecutionListener.class,
        	DirtiesContextTestExecutionListener.class,
        	TransactionalTestExecutionListener.class,
        	DbUnitTestExecutionListener.class })
        @DirtiesContext(classMode=ClassMode.AFTER_EACH_TEST_METHOD)
        @ActiveProfiles("dev")
        
        

        So application.properties can be found on target/classes, with other src/main/resources configuration files, while test configuration files can be found on target/test-classes.

        Seems that if application.properties is not in same folder of xmls, placeholders cannot be resolved..

        Reply
        • Hi Marco,

          I noticed that you use the @PropertySource annotation in your test class. I assume that you are trying to load the properties file which contains the property values used in your context configuration files. Am I correct?

          The thing is that the javadoc of the @PropertySource annotation states that it works only if it is used together with the @Configuration annotation. In other words, you should use Java configuration.

          If you want to use XML configuration, you can configure the used properties file by using the property-placeholder tag of the context namespace.

          You could also verify that the property values are found from the application.properties file after it has been copied to the target/classes directory. If this is the case, you know that the problem isn’t in your build script. If the values are not found from this file, the problem is that Maven isn’t filtering the property values from your profile specific configuration files to the application.properties file.

          By the way, if you cannot solve this problem, it would be helpful to see your build script / project. Is this possible or is it a work related project?

          Reply
          • Thank you Petri, you give me the right hint.

            Just enable in xml do the trick! So I hare removed @PropertySource annotation (that as documentation states works only if it is used together with the @Configuration annotation).

            It also works using xml files in src/integration-testing/resources, preferred over that ones defined in src/main/resources

            Thank you very much Petri!! Have a nice day

          • It also works using xml files in src/integration-testing/resources, preferred over that ones defined in src/main/resources: not true because when I run application from my eclipse workspace xml files conflict.. but I have solved appending a suffix to each xml file.

          • Hi,

            It is good to hear that you were able to solve your problem.

            I have always used the same XML configuration files for every profile (dev, test, prod, and integration-test), and moved the configuration to profile specific configuration files (although nowadays I use only Java configuration).

            On the other hand, this might not always be good enough because you might want to use different beans when you run your integration tests. I have solved this problem by using bean definition profiles (aka Spring profiles).

  • Hi Petri,
    This solved the problem I was having, very good post, thanks a lot.
    Tip: add a bonus to this entry and show how to run tests in parellel, it’s very handy and explained here http://maven.apache.org/surefire/maven-failsafe-plugin/examples/junit.html.

    Cheers
    Tom

    Reply
    • Hi Tom,

      I am happy to hear that this blog post was useful. Also, I will update this blog post when I have got time to do it and include the tip you mentioned in it. Thanks for sharing.

      Reply
  • Thanks a loooooot!!!
    you made my day!!!

    Reply
    • You are welcome! I am happy to hear that this blog post was useful to you.

      Reply
  • Dear Petri,
    I’m currently trying to generate code coverage reports Java Server for manual tests in a Maven project. I’m using eclipse with the JaCoCO plugin.

    The reports are properly generated when using automated Junit tests.

    So What I need is

    1. Run the program
    2. Perform some function on UI(Manual Testing)
    3. Exit the program
    4. Get a code coverage report for that run

    So can suggest me some way to achieve it.
    Thanks in advance.

    Best Regards,
    Krishna Singh

    Reply

Leave a Comment