Adding integration tests to a Maven build has traditionally been a bit painful. I suspect that the reason for this is that the standard directory layout has only one test directory (src/test).
If we want to use the standard directory layout and add integration tests to our Maven build, we have two options:
First, we can add our integration tests to the same directory as our unit tests. This is an awful idea because integration tests and unit tests are totally different beasts and this approach forces us to mix them. Also, if we follow this approach, running unit tests from our IDE becomes a pain in the ass. When we run tests, our IDE runs all tests found from the test directory. This means that both unit and integration tests are run. If we are “lucky”, this means that our test suite is slower than it could be, but often this means that our integration tests fail every time. Not nice, huh?
Second, we can add our integration tests to a new module. This is overkill because it forces us to transform our project into a multi-module project only because we want to separate our integration tests from our unit tests. Also, if our project is already a multi-module project, and we want to write integration tests for more than one module, we are screwed. Of course we can always create a separate integration test module for each tested module, but it would be less painful to shoot ourselves in the foot.
It is pretty clear that both of these solutions suck. This blog post describes how we can solve the problems of these solutions.
The requirements of our Maven build are:
- Integration and unit tests must have separate source directories. The src/integration-test/java directory must contain the source code of our integration tests and the src/test/java directory must contain the source code of our unit tests.
- Integration and unit tests must have different resource directories. The src/integration-test/resources directory must contain the resources of our integration tests and the src/test/resources directory must contain the resources of our unit tests.
- Only unit tests are run by default.
- It must be possible to run only integration tests.
- If an integration test fails, it must fail our build.
- The name of each integration test class must start with the prefix 'IT'.
Let’s start by creating Maven profiles for unit and integration tests.
Creating Maven Profiles for Unit and Integration Tests
First, we need to create two Maven profiles that are described in the following:
The dev profile is used in the development environment, and it is the default profile of our Maven build (i.e. It is active when the active profile is not specified). This Maven profile has two goals that are described in the following:
- It configures the name of the directory that contains the properties file which contains the configuration used in the development environment.
- It ensures that only unit tests are run when the dev profile is active.
The integration-test profile is used for running the integration tests. The integration-test profile has two goals that are described in the following:
- It configures the name of the directory that contains the properties file which contains the configuration used by our integration tests.
- It ensures that only integration tests are run when the integration-test profile is active.
We can create these profiles by adding the following XML to our pom.xml file:
<profiles> <!-- The Configuration of the development profile --> <profile> <id>dev</id> <activation> <activeByDefault>true</activeByDefault> </activation> <properties> <!-- Specifies the build.profile.id property that must be equal than the name of the directory that contains the profile specific configuration file. Because the name of the directory that contains the configuration file of the development profile is dev, we must set the value of the build.profile.id property to dev. --> <build.profile.id>dev</build.profile.id> <!-- Only unit tests are run when the development profile is active --> <skip.integration.tests>true</skip.integration.tests> <skip.unit.tests>false</skip.unit.tests> </properties> </profile> <!-- The Configuration of the integration-test profile --> <profile> <id>integration-test</id> <properties> <!-- Specifies the build.profile.id property that must be equal than the name of the directory that contains the profile specific configuration file. Because the name of the directory that contains the configuration file of the integration-test profile is integration-test, we must set the value of the build.profile.id property to integration-test. --> <build.profile.id>integration-test</build.profile.id> <!-- Only integration tests are run when the integration-test profile is active --> <skip.integration.tests>false</skip.integration.tests> <skip.unit.tests>true</skip.unit.tests> </properties> </profile> </profiles>
Second, we have to ensure that our application uses the correct configuration. When we run it by using the dev profile, we want that it uses the configuration that is used in the development environment. On the other hand, when we run our integration tests, we want that the tested code uses the configuration tailored for our integration tests.
We can ensure that our application uses the profile specific configuration by following these steps:
- Configure our Maven build to load the profile specific configuration file (config.properties) from the configuration directory of the active Maven profile.
- Configure the resource directory of our build (src/main/resources) and enable resources filtering. This ensures that the placeholders found from our resources are replaced with the actual property values that are read from the profile specific configuration file.
The relevant part of our POM file looks as follows:
<filters> <!-- Ensures that the config.properties file is always loaded from the configuration directory of the active Maven profile. --> <filter>profiles/${build.profile.id}/config.properties</filter> </filters> <resources> <!-- Placeholders that are found from the files located in the configured resource directories are replaced with the property values found from the profile specific configuration file. --> <resource> <filtering>true</filtering> <directory>src/main/resources</directory> </resource> </resources>
Let’s move on and find out how we can add extra source and resource directories to our Maven build.
Adding Extra Source and Resource Directories to Our Maven Build
The requirements of our Maven build state that our integration and unit tests must have separate source and resource directories. Because the source code and the resources of our unit tests are found from the standard test directories, we have to create new source and resource directories for our integration tests.
We can add extra source and resource directories to our build by using the Build Helper Maven Plugin. We fulfill our requirements by following these steps:
- Add the Build Helper Maven Plugin to the plugins section of our POM file.
- Configure the executions that add the new source and resource directories to our build.
First, we can add the Build Helper Maven Plugin to our Maven build by adding the following XML to the plugins section of our pom.xml file:
<plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>build-helper-maven-plugin</artifactId> <version>1.9.1</version> <executions> <!-- Add executions here --> </executions> </plugin>
Second, we have to configure the executions that add our new source and resource directories to our Maven build. We can do this by following these steps:
- Create an execution that adds the new source directory (src/integration-test/java) to our build.
- Create an execution that adds the new resource directory (src/integration-test/resources) to our build and enable resource filtering.
We can create these executions by adding the following XML to the configuration of the Build Helper Maven Plugin:
<!-- Add a new source directory to our build --> <execution> <id>add-integration-test-sources</id> <phase>generate-test-sources</phase> <goals> <goal>add-test-source</goal> </goals> <configuration> <!-- Configures the source directory of our integration tests --> <sources> <source>src/integration-test/java</source> </sources> </configuration> </execution> <!-- Add a new resource directory to our build --> <execution> <id>add-integration-test-resources</id> <phase>generate-test-resources</phase> <goals> <goal>add-test-resource</goal> </goals> <configuration> <!-- Configures the resource directory of our integration tests --> <resources> <!-- Placeholders that are found from the files located in the configured resource directories are replaced with the property values found from the profile specific configuration file. --> <resource> <filtering>true</filtering> <directory>src/integration-test/resources</directory> </resource> </resources> </configuration> </execution>
Let’s find out how we can run our unit tests with Maven.
Running Unit Tests
If we think about the requirements of our Maven build, we notice that we must take the following things into account when we run our unit tests:
- Unit tests are run only when the dev profile is active.
- When we run our unit tests, our build must run all test methods found from the test classes whose name don’t start with the prefix 'IT'.
We can run our unit tests by using the Maven Surefire plugin. We can configure it by following this steps:
- Configure the plugin to skip unit tests if the value of the skip.unit.tests property is true.
- Configure the plugin to ignore our integration tests.
We can configure the Maven Surefire plugin by adding the following XML to the plugins section of our POM file:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.18</version> <configuration> <!-- Skips unit tests if the value of skip.unit.tests property is true --> <skipTests>${skip.unit.tests}</skipTests> <!-- Excludes integration tests when unit tests are run --> <excludes> <exclude>**/IT*.java</exclude> </excludes> </configuration> </plugin>
We can run our unit tests by using the command: mvn clean test -P dev.
Let's move on and find out how we can run our integration tests with Maven.
Running Integration Tests
If we think about the requirements of our Maven build, we notice that we have to take the following things into account when we run our integration tests:
- Integration tests are run only when the integration-test profile is active.
- If an integration test fails, our build must fail as well.
- When we run our integration tests, our build must run all test methods found from the test classes whose name starts with the prefix 'IT'.
We can run our integration tests by using the Maven Failsafe plugin. We can configure it by following these steps:
- Create an execution that invokes the integration-test and verify goals of the Maven Failsafe plugin when we run our integration tests. The integration-test goal runs our integration tests in the integration-test phase of the Maven default lifecycle. The verify goal verifies the results of our integration test suite and fails the build if it finds failed integration tests.
- Configure the execution to skip integration tests if the value of the skip.integration.tests property is true.
We can configure the Maven Failsafe Plugin by adding the following XML to the plugins section of our POM file:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-failsafe-plugin</artifactId> <version>2.18</version> <executions> <!-- Invokes both the integration-test and the verify goals of the Failsafe Maven plugin --> <execution> <id>integration-tests</id> <goals> <goal>integration-test</goal> <goal>verify</goal> </goals> <configuration> <!-- Skips integration tests if the value of skip.integration.tests property is true --> <skipTests>${skip.integration.tests}</skipTests> </configuration> </execution> </executions> </plugin>
We can run our integration tests by using the command: mvn clean verify -P integration-test.
Let's move on and summarize what we learned from this blog post.
Summary
This blog post has taught us three things:
- We can add extra source or resource directories to our Maven build by using the Build Helper Maven Plugin.
- We can run our unit tests by using the Maven Surefire plugin.
- We can run our integration tests by using the Maven Failsafe plugin.
If we are writing integration tests for real life applications, we often have to use integration test specific resource files (E.g. DBUnit dataset files). We can do this by ensuring that the add-test-resource goal of Builder Helper Maven plugin is executed at Maven’s generate-test-resources lifecycle phase.
We do this by adding the following snippet to the executions section of the Builder Helper Maven plugin configuration:
This ensures that the directory src/integration-test/resources is added to our Maven build.
Great post, very helpful thanks.
I have followed your advice and added src/integration-test/resources as a test resource as above. However, this causes the integration test resources to be included in the production artifact (jar). The integration sources are correctly excluded, so I don't understand why the resources are not handled the same way. Do you have any ideas? I also asked this question on stackoverflow: http://stackoverflow.com/questions/32885696
Hi Stephen,
When I packaged the example application of this blog post, integration test resources weren't added to the created jar file. Have you modified the configuration of the Maven Jar plugin?
Hmm, will have to check this again as I'm using the maven shade plugin to build an "uber jar". Perhaps this is causing problems...
Thanks again!
Hi Stephen,
That can definitely be a problem, but I noticed that there is a way to exclude files from the "uber jar". Can you exclude the integration test resources from it?
Hi)) I have a long time trying to figure out how to learn to write integration tests, but I do not succeed. I find a lot of articles on this topic, but my head was a mess - all mixed up. Please help.
Hi,
What is the biggest problem that prevents you from achieving your goal? Also, what framework (Spring, Java EE, and so on) are you using?
Hey Petri,
Very nice article.
It was very helpful for me!!!
Thanks a lot.
Best Regards,
Celso
You are welcome! I am happy that I could help you out.
Hi Petri,
I want to use Jacoco for web application code coverage.
I think the above only works fine if you are running tests and Web Server on the same machine.
Currently my Jenkins run on one machine and my server is on another machines.Does it still work for that scenarior or what changes should I do for that.
Hi Rang,
If you are referring to this blog post, I have to confess that I have not tried running the example with Jenkins yet.
I am planning to do this at some point because I want to make sure that it works. In the meantime, you might want to take a look at this blog post which describes the usage of the JaCoCo Jenkins plugin.
Jacoco has a tcpmode option that can operate as either client or server. From the Jenkins CI - you can pull the coverage metrics and reset the collection. So as long as you start the server with the appropriate agent configuration you will be able to collect coverage data remotely (be advised ther is no security here - so one would only do this in a trusted environment)
Thanks for the tip! I will try this out when I have got some time to do it and write a new blog post about it.
Very helpful post. Thanks.
You are welcome!
It would sure be nice if we didn't have the magic name requirement (**/IT*.java) for excluding integration tests from the unit test build. Do you know of a way to accomplish this?
Yes, that would be nice.
One option is to move the configuration of the Build Helper Maven plugin to the profile which you use to run the integration tests. This way the integration tests would be added to the classpath only when you use the correct profile which means that you can name your tests anyway you want.
I haven't tried this out but it should work (in theory).
The trouble with this is that maven includes `src/test/java` by default and there seems no way of preventing this. So `**/IT*java` or similar is required. After having used Gradle and returning to Maven, I'm understanding why many devs don't like Maven.
Actually I found a technique that doesn't require any naming conventions for unit and integration tests. Check out this blog post that explains how you can create unit and integration tests suites by using JUnit categories.
Hello, great contribution Petri.
I used this configuration in a multi module project and it worked ok.
The only thing is that using Eclipse (Kepler + Maven 3) I found that the eclipse plugin had a problem with the execution declaration in the maven helper plugin.
If this happens to anyone else, here are the three possible solutions:
1. In the quick fix (of eclipse) select the option of adding the ignore of the execution to the eclipse build in the eclipse project file. Even though it warns that it´s in an experimental fase, it works fine.
2. In the quick fix (of eclipse) select the option of adding the ignore to the eclipse build in the project POM.
3. Update the m2 lifecycle mapping connector. It connects to the marketplace and installs it. Restart eclipse and the problem is solved.
These three approaches are independent.
Hope this help to somebody.
Hi Emiliano,
It is nice to know that this tutorial was useful to you. Also, thank you for sharing the solutions for the problem you had with Eclipse and Build Helper Maven plugin. I am sure that this information is useful to someone!
Approaches 1 & 2 deprive you of a properly setup .classpath (i.e. src/integration-test/java appears as a source folder in Eclipse).
BTW For the latest version of 'm2e connector for build-helper-maven-plugin' see this thread http://comments.gmane.org/gmane.comp.ide.eclipse.plugins.m2eclipse.user/10082
Thank you for leaving this comment! I am sure that it is useful for Eclipse users.
Hello Petri,
Very informative and nice article.
Need suggestion: I am planning to write integration test for my product.
Product have 3 different web applications with 3 different spring configuration. Currently, Integration test is written to load just base configuration but I want to separate modules(applications) and write common integration test across modules ..so planing to create 4 modules like one for base and rest 3 like application specific modules so each specific module will load their own spring configuration while doing integration test.
What do you think will be best approach? Your help and views will be greatly appreciated.
Many Thanks
Hi Tej,
I would configure the application context in each web application module (no configuration in the core module). The reason for this is that if you have to override the base configuration, you can run into really weird problems. I have to admit that my approach means that you probably have to create similar application context configuration classes / files in each web application module but I am ready to make that tradeoff.
Also, I would simply write integration tests for each web application. This way I could ensure that the base module is working correctly with each web application.
I hope that this answered to your question. If you have any additional questions, feel free to ask them!
This is by far one of the best posts I've read on Integration testing. The content is very well presented too.
Keep up the great work.
Thanks! I am happy to hear that this blog post was useful to you.
Greetings,
Very nice article! However, I am not quite being able to pull this off.
"mvn clean verify -P integration-test" is correctly executing only the TestOneIT.java I have in my src/it/java folder
However, the "mvn clean test " is executing my jUnit tests inside src/test/java AND my TestOneIt.java as well :( Like the if the ${skip.integration.tests}
inside the maven-failsafe-plugin isn't doing nothing.
Help?
Sorry, my bad. the problem was the search pattern for the integration test files.
It was just a matter of changing **/IT*.java to **/*IT*.java
No worries. Good to hear that you were able to solve your problem!
good one :)
Thanks! I am happy to hear that you liked this blog post.
Hi Petri, I'm working in a multi-module setting, I have:
pom.xml
- persistence
- app-services
- rest
Each of these modules have both unit and integration tests, and I was wondering if I can just declare all of these profile settings in the parent pom? The reason I'm not sure is because of the filtering, the parent won't have any resources to filter (it's just a "pom")
I know this is a candidate for TIAS, but I figured if you knew off the top of your head it would save me time mocking up a bunch of tests.
Interesting question. If I remember correctly, I have never done this, but I thought that it would be a walk in the part. It seems that I was wrong. I was able to find three "competing" solutions to this problem:
I haven't tested any of these solutions but if I would have to do it, I would start by using the third solution and move my way back up.
Hi Petri and thank you for your very helpful tutorial!
I'm trying to implement integration-testing in the way you describe. In my situation I also need to keep test .xml context configuration files separated by ones of application. So I put them into /src/integration-test/resources/META-INF.
Looking in target folder I have found that: files in src/main/resources are placed under PROJECT/target/classes/, while files in /src/integration-test/resources/ are placed in PROJECT/target/test-classes
application.properties in PROJECT/target/classes is processed by maven plugins that replace ${placeholder} with value of ${profile}/config.properties, while application.properties in PROJECT/target/test-classes is not processed. So executing maven integration-test I get exception like: Circular placeholder reference 'db.driver' in property definitions.
Trying to put application.properties without placeholders in src/integration-test/resources seems than application.properties is not loaded by @PropertySource("classpath:application.properties"). I get exception like: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set.
So I'm wrong in setting my test context, or I need to modify my pom to reach my requirements?
Another question is, what if I want to execute/debug each single test from eclipse?
Thank you very much Petri!
Have a good day
Hi Marco,
The idea behind the configuration explained here is that you can use the same properties file (src/main/resources/application.properties) in your application and in your tests because Maven selects the actual property values based on the active profile.
In other words,
This means that you don't need to create a separate properties file to the src/integration-test/resources directory because you can use the properties file found from the src/main/resources directory. This file contains the test specific configuration as long as the correct profile (integration-test) is active.
You can the refer to this file in your test specific application context configuration file like this: 'classpath:application.properties'.
On the other hand, if you want to create a separate properties file for your integration tests, you shouldn't name it application.properties because if you do, the classpath has two files called application.properties and this can cause problems if you refer to them by using the notation: 'classpath:application.properties' (the classloader loads one of them but you cannot know which one).
I hope that this shed some light to your properties file related problems.
I haven't used Eclipse for six years but if I have to run / debug a single integration test in my IDE (IntelliJ Idea), I follow these steps:
I hope that this answered to your questions. If not, let me know!
Thank you Petri for your reply!
I have tryed to use /src/main/resources/application.properties in conjunction with .xml configuration files in /src/intergation-test/resources/META-INF/*-test.xml (I have renamed them with test suffix) but context initialization fails: placeholders cannot be found.
Here is how test is configured:
So application.properties can be found on target/classes, with other src/main/resources configuration files, while test configuration files can be found on target/test-classes.
Seems that if application.properties is not in same folder of xmls, placeholders cannot be resolved..
Hi Marco,
I noticed that you use the
@PropertySource
annotation in your test class. I assume that you are trying to load the properties file which contains the property values used in your context configuration files. Am I correct?The thing is that the javadoc of the
@PropertySource
annotation states that it works only if it is used together with the@Configuration
annotation. In other words, you should use Java configuration.If you want to use XML configuration, you can configure the used properties file by using the
property-placeholder
tag of thecontext
namespace.You could also verify that the property values are found from the application.properties file after it has been copied to the target/classes directory. If this is the case, you know that the problem isn't in your build script. If the values are not found from this file, the problem is that Maven isn't filtering the property values from your profile specific configuration files to the application.properties file.
By the way, if you cannot solve this problem, it would be helpful to see your build script / project. Is this possible or is it a work related project?
Thank you Petri, you give me the right hint.
Just enable in xml do the trick! So I hare removed @PropertySource annotation (that as documentation states works only if it is used together with the @Configuration annotation).
It also works using xml files in src/integration-testing/resources, preferred over that ones defined in src/main/resources
Thank you very much Petri!! Have a nice day
It also works using xml files in src/integration-testing/resources, preferred over that ones defined in src/main/resources: not true because when I run application from my eclipse workspace xml files conflict.. but I have solved appending a suffix to each xml file.
Hi,
It is good to hear that you were able to solve your problem.
I have always used the same XML configuration files for every profile (dev, test, prod, and integration-test), and moved the configuration to profile specific configuration files (although nowadays I use only Java configuration).
On the other hand, this might not always be good enough because you might want to use different beans when you run your integration tests. I have solved this problem by using bean definition profiles (aka Spring profiles).
Hi Petri,
This solved the problem I was having, very good post, thanks a lot.
Tip: add a bonus to this entry and show how to run tests in parellel, it's very handy and explained here http://maven.apache.org/surefire/maven-failsafe-plugin/examples/junit.html.
Cheers
Tom
Hi Tom,
I am happy to hear that this blog post was useful. Also, I will update this blog post when I have got time to do it and include the tip you mentioned in it. Thanks for sharing.
Thanks a loooooot!!!
you made my day!!!
You are welcome! I am happy to hear that this blog post was useful to you.
Dear Petri,
I'm currently trying to generate code coverage reports Java Server for manual tests in a Maven project. I'm using eclipse with the JaCoCO plugin.
The reports are properly generated when using automated Junit tests.
So What I need is
1. Run the program
2. Perform some function on UI(Manual Testing)
3. Exit the program
4. Get a code coverage report for that run
So can suggest me some way to achieve it.
Thanks in advance.
Best Regards,
Krishna Singh
Hi Krishna,
I have never done this myself but it should be possible.
Thanks a lot. It's very helpful for me!
You are welcome! I am happy to hear that this blog post was useful to you.
Hi,
Thank you for this great article. I am completely new to this world but voila! i was able to work with the steps mentioned above. However I am facing 1 issue.
maven while building is looking for config.properties under {basedir}\profiles\* than {basedir}\src\main\profiles.
Any clue?
If you want to change the location of the
profiles
directory, you have to change the value of thefilter
element.For example, if you want to put the profile specific configuration directories to the
src/main/profiles
directory, you have to use the following configuration:If you want to get more information about this, you should read this blog post.
Excellent work! Many thanks Petri.
You are welcome! I am happy to hear that this blog post was useful to you.
Thanks Petri for very useful article.
I've a question on integration tests: I need to run the same integration tests several times on a web application in different configurations. For example, a web app recognises propertyA, I want to integration test the web app with propertyA=1234 and propertyA=4321.
How would you recommend to do that?
Hi Fedor,
Thank you for your kind words. I really appreciate them.
Do you think that you could create a controller method that is used to set the value of the
propertyA
before your integration tests are run? Of course you would have to ensure that this method can be invoked only when the integration tests are run, but this shouldn’t be a problem.I know that this sounds like a hack, but if you want to change the value of that property without using different Maven profiles for each legal property value, this is the easiest way to do it. If you find another way, let me know!
Verify helpful and well written. Thanks!
You are welcome! I am happy to hear that this blog post was useful to you.
Hi Petri,
thanks for the excellent article. Are you using JUnit 4.8? If so, there is a nice way to filter using a JUnit category and an annotation: http://www.java-tutorial.ch/maven/maven-integration-test
I also decided to keep integration testing associated with the maven integration-test lifecycle phase. The only downside is that it runs the test phase first, but that can be inhibited using a variable like so:
org.apache.maven.plugins
maven-surefire-plugin
2.17
${skip.unit.tests}
com.my.junit.marker.IntegrationTest
Hi Adam,
Thank you for sharing this. Although I prefer keeping my integration and unit tests in different directories, the
@Category
annotation helps me to get rid of the stupidIT
prefix (or suffix)!Also, I think that it might me help to implement a few other use cases as well. I have to take a closer look at it and figure out if I can create tests that are invoked only when the build is run on a CI server.
Hi Petri,
Thank you for your tutorial.
In my case I have OSGI plugin tests and unit tests in one OSGI fragment project, but I want to execute them separately using approach in your tutorial. So basically between tycho-surefire and maven-surefire plugin. Is it possible ?
Thanks & Regards
SetNug
Hi SetNug,
Unfortunately I don't know the answer to your question since I have never used OSGI :( However, I found some resources that might be useful to you:
I hope that these resources help you to solve your problem.
Hi Petri,
Thank you very much. Your links are very helpful.
Regards,
Setya
Hi Petri,
Great article, it really helped a lot.
I have more problem statement where we have two different standalone projects ( Project A and Project B )
Project A is a application which has all the code and can be deployed on different servers.
Project B have all the tests written which will test Project A
I want to find a way such that i can find Code coverage for project A by running tests from project B. ( Both the projects are maven projects )
I have written a blog post that describes how you can create a code coverage report for unit and integration tests by using the JaCoCo Maven plugin. That post uses a bit old version of the JaCoCo Maven plugin, but the newer version has some goals that could help you out.
Hi,
I am relatively new to maven and TDD. I am reading the book - Growing Object Oriented Software Guided by Tests. The project explained in the book keeps unit tests, integration tests and end-to-end tests (all in java) in three different test-source directories. The project explained in the book uses Ant to build the project. I am exploring ways to accomplish the same in maven. That is how I ran into your post about integration testing with maven.
I followed most of the items described in this post. But I would like to clarify the following items with you:
1. Is it mandatory to create profile specific config files? Will they be used only when there are profile specific settings that need to be applied?
2. If I want to add one more profile for end-to-end tests, use a separate test directory for them and run them independent of unit and integration tests, will I be able to achieve it using maven? If I can perform this task using maven, can you please give me some tips to accomplish it?
Thanks,
Richards.
Hi,
I think I should address my second query in a different manner. May be I should treat end-to-end tests as integration tests itself. I think I should look at the possibility of adding end-to-end tests into a separate test-source directory but allow them to execute along with integration tests. Is there a better approach?
Thanks,
Richards.
Hi Richards,
I am sorry that it took me some time to answer to your comment.
It is not mandatory to create these configuration files, but if you decide to not create them, you shouldn't configure them in your POM file. Otherwise your build will fail because Maven cannot find the missing the configuration files.
You can definitely run end-to-end tests with Maven even if you have unit and integration tests as well. Your second question (running e-2-e tests together with integration tests) is quite interesting because I think that a good build script (and test suite) gives you the possibility to select the invoked tests. This is important because you want to run different tests in different situations.
For example, if you are a developer, you might want to run only unit tests because integration and e-2-e tests are slow.
On the other hand, if you are building the project on CI server, you can create different builds (every commit, daily etc) and select the invoked tests based on the build type. For example, it might no be practical to run all tests for every commit because getting the feedback takes too long.
If you have any additional questions, don't hesitate to ask them.
Hi Petri,
Thank you for your reply.
"You can definitely run end-to-end tests with Maven even if you have unit and integration tests as well."
- If my end-to-end tests are also written in java (for a simple application) and stored under /src/java/e2e/ directory, can you please tell me how they can be executed independent of unit and integration tests? I am not sure how to configure maven-surefire-plugin and maven-failsafe-plugin to meet this requirement.
Thanks,
Richards.
Hi Richards,
Do you want to run your integration tests and end-to-end tests at the same time, or do you want to create different profiles for them (i.e. it is possible to run either integration or end-to-end tests)?
Hi Petri,
Sorry for the delay to respond to your question.
I want to run integration tests and end-to-end tests separately. Yes, I would like to create different profiles for them.
Thanks,
Richards.
Hi Richards,
If I remember correctly, you can use a profile specific plugin configuration for this purpose. You can do this by following these steps:
If you don't know how to do this, let me know and I might write a blog post about this.
Is a test requiring an external database and no web layer considered an integration test ? Or does it need to involve a web layer to be considered an integration test ?
Hi Stephane,
I think that you just stirred a hornet's nest. There are a lot of people out there who cannot agree what these terms (unit and integration test) means. That being said, I think that the test you described is an integration test because it uses a database (either an external database or an in-memory database).
Hi Petri,
Great article - very helpful!
The solution you presented prevents `-DskipTests` from skipping all tests. You need to do `-Dskip.unit.tests -Dskip.integration.tests`. Do you know of a way to get the former working? I played with this for a while and gave up.
My solution was to not use the two switches. And I added an `**/IT*java` to the Unit Tests surefire and added a dummy `surefire` with `**/*java` to the integration-tests profile to prevent the unit tests running. Another difference is that my unit test surefire was outside of any profile - I'm not sure if this would make a difference in the case of your multi-profile configuration.
Cheers,
Brooke
Hi Brooke,
Ah. I have to admit that I have never thought of that because I haven't needed that "feature". I would probably just create a profile called: skipTests and configure it to skip both unit and integration tests (set the values of the
skip.integration.tests
andskip.unit.tests
properties totrue
). This way I wouldn't have to make other changes to the configuration. However, I admit that I have no idea if this is the best way to skip all tests in this case.Hey Petri,
Can I configure Jacoco in Tomcat. I tried searching all the available information on google, but there is no data available which gives stepwise information to configure JaCoCo with Tomcat. Can you please help?
Hi,
It should be possible to do this (at least in theory). Unfortunately I have never done this myself. In other words, I don't know how you can do it.
Hi Petri,
Thanks for a nice blog, but in my case the dev and test code are in different repos, and i want to get code coverage of dev code by running Test Suite(Test Rest API services) to get my code coverage of dev code, is there any way i can instrument dev code first and then run test suite against it.
Thanks,
Monis.
Hi,
the JaCoCo agent must be running when the tests are run because this agent records that data that reveals which parts of the code were run by your tests.
Also, if you want to deploy your code into an external server before you run your tests, you need to figure out a way to record this data on the server, copy it back to computer that runs the tests, and create the code coverage report. Unfortunately I don't know how you can create a build script that does this.
This webpage is wonky.The characters are rendering as
"&lt;" (minus the quotations) for the
All the other pages on your site show up fine.
Oops. I fixed the problem. Thank you for pointing this out!
I have put my integration and unit test in the same folder like src/test/.../*IT.java.
and I am executing the failsafe-plugin to run the integration test, but while displaying this in Jenkins as a post build action it is not able to publish in report, it is only giving the Unit test report not the integration test report.
Please suggest.
Hi Petri,
Thank you for this blog - its great - I have followed it on my own work project but I am having an issue which I have log on SO now,
https://stackoverflow.com/questions/55654489/how-to-stop-build-helper-maven-plugin-running-during-dev-goals-executions
I am finding that when I run the Compile/Test goals for the DEV profile - the config.properties file for the Integration-Test directories is also being copied and over-writting the one from Dev-Test. I am not sure how to control this.
Hi, Petri,
per your suggestion , if project is a multi-module project, and we want to write integration tests for more than one module, it's better to create a separate integration test module for all tested module, is it right?
Hi Anson,
Thank you for your question. Sadly, I have to answer that it depends from your personal preferences, the requirements of your CI process, the used build tool, and the used testing framework.
However, if I am writing tests with JUnit 5 and I use Maven, I tend to follow these two rules:
If you have any additional questions, don't hesitate to ask them.