Getting Started With Gradle: Integration Testing With the TestSets Plugin

My previous blog post described how we can add integration tests to our Gradle build.

A few days after I had published that blog post, I learned that we don’t have to do everything from the scratch (thanks David) because we can simplify our build script by using the Gradle TestSets plugin.

This blog post describes how we can add integration tests to our Gradle build, remove the unnecessary clutter from our build.gradle file, and fulfil the requirements specified in my previous blog post.

Let’s get started.

The example application of this blog post is tested with Gradle 4.6.

Additional Reading:

If you are not familiar with Gradle, you should read the following blog posts before you continue reading this blog post:

Applying the Gradle TestSets Plugin

Before we can use the Gradle TestSets plugin, we have to apply it. We can do this by following this two-step process:

We can use the Gradle TestSets plugin only if we also using the java and/or groovy plugin.

First, we have to configure the dependencies of our build script. We can do this by following these steps:

  1. Configure Gradle to use the Bintray’s JCenter Maven repository when it resolves the dependencies of our build script.
  2. Add the TestSets plugin dependency to the classpath configuration.

The relevant part of our build.gradle file looks as follows:

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.4.5'
    }
}

apply plugin: 'application'
apply plugin: 'java'

Second, we have to apply the TestSets plugin. After we have done this, the relevant part of our build.gradle file looks as follows:

buildscript {
    repositories {
        jcenter()
    }
    dependencies {
        classpath 'org.unbroken-dome.gradle-plugins:gradle-testsets-plugin:1.4.5'
    }
}

apply plugin: 'application'
apply plugin: 'java'
apply plugin: 'org.unbroken-dome.test-sets'
If you are using Gradle 2.1 or newer, and you want to use the new plugins DSL, you can apply the Gradle TestSets plugin by adding the following snippet to your build.gradle file:

plugins {
    id 'org.unbroken-dome.test-sets' version '1.4.5'
}

Additional Reading:

Let’s move on and find out how we can add a new test set to our Gradle build.

Adding a New Test Set to Our Build

We can create a new test set called integrationTest by adding the following snippet to our build.gradle file:

testSets {
    integrationTest
}

After we have done this, the TestSets plugin adds the following "features" to our Gradle build:

  • It adds a source set named integrationTest to our build. This means that the src/integrationTest/java directory contains the source code of our our integration tests and the src/integrationTest/resources directory contains the resources of our integration tests.
  • It adds two new dependency configurations to our build.
    • The integrationTestCompile dependency configuration contains the dependencies that are required to compile our integration tests. It also contains all dependencies that are required to compile our unit tests.
    • The integrationTestRuntime dependency configuration contains the dependencies that are needed when our integration tests are run. It also contains all dependencies that are required to compile our integration tests and to run our unit tests.
  • It adds two new tasks to our Gradle build:
    • The integrationTest task will run our integration tests.
    • The integrationTestJar task will package our integration tests to a jar file.

However, there is still one problem left. The requirements of our Gradle build states that:

  • The integration tests must be found from the src/integration-test/java directory.
  • The resources of our integration tests must be found from the src/integration-test/resources directory.

In other words, we have to change the directory name from integrationTest to integration-test. We can do this by specifying the value of the dirName property. We can specify that value by adding the following snippet to our build.gradle file:

testSets {
    integrationTest { dirName = 'integration-test' }
}
If you are using Android Studio and you want to navigate between the integration test code and the code under test, you can define your source set in a way that is described in this comment.

Let’s move on and add some dependencies to our new dependency configurations.

Declaring the Dependencies of Our Integration Tests

We can now add dependencies to the integrationTestCompile and integrationTestRuntime dependency configurations. For example, because the integration tests of our example applications use AssertJ 3.0, we have to add the assertj-core dependency to the integrationTestCompile configuration. After we have done this, the dependencies build script block of our build.gradle file looks as follows:

dependencies {
    compile 'log4j:log4j:1.2.17'
    testCompile 'junit:junit:4.11'
    integrationTestCompile 'org.assertj:assertj-core:3.0.0'
}

Let’s move on and find out how we can configure the task that runs our integration tests.

Configuring the Integration Test Task

Although the Gradle TestSets plugin created the integrationTest task when we created our test set, we have to make some changes to its configuration if we want to fulfil the requirements of our Gradle build.

To be more specific, we have to ensure that unit tests are run before integration tests and that integration tests are run when we invoke the build task. We can do this following these steps:

  1. Ensure that our integration tests are run before the check task and that the check task fails the build if there are failing integration tests.
  2. Ensure that our unit tests are run before our integration tests. This guarantees that our unit tests are run even if our integration tests fails.

We can do these configuration changes by adding the following lines to our build.gradle file:

check.dependsOn integrationTest
integrationTest.mustRunAfter test
Gradle skips tasks whose input and output are up to date. This means that if we run our integration tests twice without making any changes, our integration tests are run only once. Because we did not make any changes before we tried to run them for the second time, Gradle skips the integrationTest task because its input and output are up to date.

If we want to run our integration tests every time when the integrationTest task is invoked, we have to tell Gradle that the outputs of the integrationTest task should always be considered out of date.

A Gradle task has a property called outputs, and the type of this property is TaskOutputs. If we want that the outputs of the integrationTest task are always considered out of date, we have to ensure that that the upToDateWhen() method of the TaskOutputs interface always returns false. We can do this by adding the following code to our build.gradle file:

project.integrationTest {
    outputs.upToDateWhen { false }
}

Additional Reading:

Let’s move on and find out how we can run our unit and integration tests.

Running Our Unit and Integration Tests

We have now created a new test set and configured the task that runs our integration tests. This means that we can finally run both unit and integration tests. The requirements of our build states that we must be able to run only unit tests, only integration tests, or all tests.

First, if we want to run only unit tests, we can use one of these two options:

  • We can run our unit tests by running the command: gradle clean test at the command prompt.
  • We can run our build and exclude integration tests by running the command: gradle clean build -x integrationTest at the command prompt.

Second, if we want to run only integration tests, we can choose one of the following options:

  • We can run our integration tests by running the command: gradle clean integrationTest at the command prompt.
  • We can run our build and exclude unit tests by running the command: gradle clean build -x test at the command prompt.

Third, if we want to run all tests, we can use one of these two options:

  • We can run our unit and integration tests by running the command: gradle clean test integrationTest at the command prompt.
  • We can run our build by running the command: gradle clean build at the command prompt.

When we run our tests, Gradle creates the HTML reports of our unit and integration tests to the following directories:

  • The build/reports/tests/integrationTest directory contains the HTML report that contains the test results of our integration tests.
  • The build/reports/tests/test directory contains the HTML report that contains the test results of our unit tests.

Let’s move on and summarize what we learned from this blog post.

Summary

This blog post has taught us four things:

  • When we create a new test set, the Gradle TestSets plugin creates a new source set, creates dependency configurations for the created test set, and creates the task that runs the tests found from the created test set.
  • We can configure the directory name of the created test set by setting the value of the dirName property.
  • We can configure the order in which our tasks are invoked.
  • We can exclude tasks by using the -x command line option.

P.S. You can get the example application of this blog post from Github.

If you want learn how to use Gradle for running your unit, integration, and end-to-end tests, which ensure that your Spring web application is working as expected, take a look at my upcoming Test With Spring Course.
12 comments… add one
  • B. K. Oxley (binkley) Dec 22, 2015 @ 17:10

    See github issue #12 (https://github.com/unbroken-dome/gradle-testsets-plugin/issues/12) for more on testsets and patching the output dirs for test output and reports. In your example for HTML reports you might want to bring in the extra 2 lines to fix the binary output dirs also.

    • Petri Dec 22, 2015 @ 19:51

      Thank you for reporting this problem. I will take a look at it after Christmas.

  • Jason Harrison Jan 17, 2017 @ 20:05

    Thank you for your tutorial. One addition will make it even better is to add a sourceSets definition so that you can navigate in Android Studio between the integration test code and the code under test:

    
    sourceSets {
        integrationTestCompile {
            java {
                srcDir 'src/integrationTest/java'
            }
            resources {
                srcDir 'src/integrationTest/resources'
            }
            compileClasspath += sourceSets.main.runtimeClasspath
        }
    }
    
    
    • Petri Jan 26, 2017 @ 22:59

      Hi Jason,

      Thank you for the tip. I will a new note to the actual post that contains a link to your comment. I am sure that it will be helpful to my readers.

  • will Mar 6, 2017 @ 5:49

    Thank you for the excellent outline. Your blogs are always helpful.

    I have a question because I'm stuck finding an answer that actually works to date. As you know the default run-time directory for tests is to run in the project or sub-project directory. That is in the same location as my build.gradle file.

    How can I set-up a test run-time directory? There is some documentation under the test DSL that talks about the woringDir property. When I set a new value, it does nothing.

    * https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html#org.gradle.api.tasks.testing.Test:workingDir

    I think I can use profiles which would be a better approach (conceptually) if it doesn't make things messy over several sub-projects. Any ideas? Am I looking in the wrong place(s)?

    Thanks,
    Will

    • Petri Mar 8, 2017 @ 9:20

      Hi Will,

      Unfortunately I am a Gradle newbie myself (and these basically document my learning process). That is why I cannot answer to your question. I hope that you can solve your problem though.

  • Joey Mar 10, 2017 @ 16:10

    Hey Petri,
    first let me thank you for this tutorial. I have one question though: How I'm able to run only a certain testmethod of a testclass? With gradles test scope I'm able to run a single testmethod via:
    gradle test --tests *TestClass.testMethod
    Is there a way to do something similar within the integrationTest scope?
    Thanks,
    Joey

    • Joey Mar 10, 2017 @ 16:18

      My fault - this is indeed possible with the integrationTest scope as well. Maybe you could add this to the "Running our Unit and Integration Tests" section.

  • Ivan Mar 28, 2017 @ 21:02

    Hi Petri.
    Thank you for your very informative post.
    Actually I am facing an issue and I wanted to know if this has happened to you before:
    I am able to run integration tests using gradle integrationTest task, but unfortunately I can not run integration tests classes directly from my java file, I suspect, the JUnit test runner can not find somehow my tests since I get this error : Class not found: "com.bezirk.u.plugin.retail.ProductCategoryInterestPosPluginIntegrationTest"Empty test suite."

    • Petri Mar 29, 2017 @ 9:19

      Hi Ivan,

      You are right. It seems that your IDE cannot find your test methods for some reason. Which IDE and Gradle version are you using? Also, I haven't seen this myself, but I have to also admit that I have used Gradle only in my personal hobby projects that have fairly simply build scripts.

Leave a Reply