Running Kotlin Tests With Maven

When we start a new project with Kotlin, one of the first things that we have to do is to create a Maven project that can compile and run our application. However, our job isn't done yet. We still have to figure out a way to compile and run our automated tests. This blog post helps us to create a Maven project that can compile and run both unit and integration tests which use JUnit 5.

After we have finished this blog post, we:

• Can create a Maven project that can compile unit and integration tests which use Kotlin.
• Know how we can add custom source and resource directories to our Maven build.
• Understand how we can skip either unit or integration tests.
• Know how we can filter the invoked tests with JUnit 5 tags.
• Can run both unit and integration tests with Maven.

Let's start by taking a look at the requirements of our Maven build.

The Requirements of Our Maven Build

The requirements of our Maven Build are:

• All code (application and tests) must use Kotlin.
• Unit and integration tests must have separate source and resource directories.
• It must be possible to run only unit or integration tests.
• It must be possible to run both unit and integration tests.
• Only unit tests are run by default.

Next, we will take a look at the system under test.

Introduction to System Under Test

During this blog post we will write both unit and integration tests for the MessageService class. This class has one method called getMessage() which returns the message: 'Hello World!'.

The source code of the MessageService class looks as follows:

class MessageService {

    fun getMessage(): String {
        return "Hello World!"
    }
}

Let's move on and find out how we can create a simple Kotlin project that can run only unit tests.

Running Unit Tests With Maven

When we want to create a Kotlin project that can run only unit tests, we have to follow these steps:

First, we have to specify three properties:

• The kotlin.version property specifies the used Kotlin version.
• The kotlin.compiler.incremental property enables incremental compilation.
• The junit-jupiter.version property specifies the used JUnit 5 version.

After we have specified these properties, the properties section of our POM file looks as follows:

<properties>
	<kotlin.version>1.3.21</kotlin.version>
	<kotlin.compiler.incremental>true</kotlin.compiler.incremental>
	<junit-jupiter.version>5.4.2</junit-jupiter.version>
</properties>

Second, we have to specify the required dependencies. Because we want to write tests for a Kotlin application by using Kotlin and JUnit 5, we have to specify the following dependencies:

• The Kotlin standard library (kotlin-stdlib).
• AssertJ Core.
• The JUnit 5 dependencies (JUnit Jupiter API and JUnit Jupiter engine).

After we have specified the required dependencies, the dependencies section of our POM file looks as follows:

<dependencies>
    <dependency>
        <groupId>org.jetbrains.kotlin</groupId>
        <artifactId>kotlin-stdlib</artifactId>
        <version>${kotlin.version}</version>
    </dependency>

    <!-- Testing Dependencies -->
    <dependency>
        <groupId>org.assertj</groupId>
        <artifactId>assertj-core</artifactId>
        <version>3.12.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-api</artifactId>
        <version>${junit-jupiter.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>${junit-jupiter.version}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Third, we have to configure the source and test directories of our Maven build. We will use the following directories:

• The src/main/kotlin directory contains the source code of our application.
• The src/test/kotlin directory contains the source code of our unit tests.

After we have configured these directories, the build section of our POM file looks as follows:

<build>
    <sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
    <testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
</build>

Fourth, we have to ensure that Maven compiles our Kotlin code. We can do this by following these steps:

1. Add the Kotlin Maven plugin to our Maven build.
2. Create an execution that invokes the compile goal of the Kotlin Maven plugin.
3. Create an execution that invokes the test-compile goal of the Kotlin Maven plugin.

After we have configured the Kotlin Maven plugin in the plugins section of our POM file, its configuration looks as follows:

<plugin>
    <artifactId>kotlin-maven-plugin</artifactId>
    <groupId>org.jetbrains.kotlin</groupId>
    <version>${kotlin.version}</version>

    <executions>
        <execution>
            <id>compile</id>
            <goals>
                <goal>compile</goal>
            </goals>
        </execution>

        <execution>
            <id>test-compile</id>
            <goals>
                <goal>test-compile</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Fifth, we have to ensure that Maven runs our unit tests are run by using the Maven Surefire plugin (version 2.22.1). We can do this by adding the following snippet to the plugins section of our POM file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.1</version>
</plugin>

We have now configured our Maven build. However, before we can run our unit tests, we have to write one test class and put this class to the src/test/kotlin directory. The source code of our test class looks as follows:

import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.DisplayName
import org.junit.jupiter.api.Test

class MessageServiceTest {

    private lateinit var service: MessageService

    @BeforeEach
    fun configureSystemUnderTest() {
        service = MessageService()
    }

    @Test
    @DisplayName("Should return the correct message")
    fun shouldReturnCorrectMessage() {
        val message = service.getMessage()
        assertThat(message).isEqualTo("Hello World!")
    }
}

We can now run our unit tests by using the command: mvn clean test. When we run this command, we should see that the Maven Surefire plugin runs our unit test:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.testwithspring.master.kotlin.MessageServiceTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 s - 
in com.testwithspring.master.kotlin.MessageServiceTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

So, we have just created a Maven project that can compile and run our unit tests. However, our build doesn't have any support for integration tests. Next, we will find out how we can support both unit and integration tests.

Running Unit and Integration Tests With Maven

Because we want to run both unit and integration tests with Maven, we have to make some changes to our Maven build. Let's start by configuring the source and resource directories of our Maven build.

Configuring the Source and Resource Directories

We can configure the source and resource directories of our unit and integration tests by following these steps:

First, we have add the resource directory of our integration tests to our Maven build. We can do this by using the Build Helper Maven plugin. When we want to use this plugin, we have to follow these steps:

1. Add the Build Helper Maven plugin to our Maven build.
2. Create an execution that adds the src/integration-test/resources directory to our Maven build.

After we have configured the Build Helper Maven plugin, its configuration looks as follows:

<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>3.0.0</version>
    <executions>
        <execution>
            <id>add-integration-test-resources</id>
            <phase>generate-test-resources</phase>
            <goals>
                <goal>add-test-resource</goal>
            </goals>
            <configuration>
                <resources>
                    <resource>
                        <filtering>true</filtering>
                        <directory>src/integration-test/resources</directory>
                    </resource>
                </resources>
            </configuration>
        </execution>
    </executions>
</plugin>

Second, because our unit and integration tests have separate source directories, we have to ensure that Maven can compile the source code of our unit and integration tests. We can do this by following these steps:

1. Remove the testSourceDirectory element from the build section of our POM file.
2. Ensure that the Kotlin Maven plugin can compile the source code found from the src/test/kotlin and src/integration-test/kotlin directories.

After we have made these changes to our Maven build, the configuration of the Kotlin Maven plugin looks as follows:

<plugin>
    <artifactId>kotlin-maven-plugin</artifactId>
    <groupId>org.jetbrains.kotlin</groupId>
    <version>${kotlin.version}</version>
    <executions>
        <execution>
            <id>compile</id>
            <goals>
                <goal>compile</goal>
            </goals>
        </execution>
        <execution>
            <id>test-compile</id>
            <goals>
                <goal>test-compile</goal>
            </goals>
            <configuration>
                <sourceDirs>
                    <sourceDir>${project.basedir}/src/test/kotlin</sourceDir>
                    <sourceDir>${project.basedir}/src/integration-test/kotlin</sourceDir>
                </sourceDirs>
            </configuration>
        </execution>
    </executions>
</plugin>

After we have configured the source and resource directories of our unit and integration tests, we have to ensure that we can run our integration tests Maven.

Running Integration Tests With Maven

We can run our integration tests with Maven by using the Maven Failsafe plugin (version 2.22.1). We can add this plugin to our Maven build by adding the following snippet to the plugins section of our POM file:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.22.1</version>
</plugin>

After we have added the Maven Failsafe plugin to our Maven build, we have to configure it by following these steps:

1. Ensure that the Maven Failsafe plugin runs all test classes whose name ends with the suffix: Test. We can do this by changing the wildcard pattern that is used to select the invoked test classes.
2. Ensure that the Maven Failsafe plugin runs both integration-test and verify goals. The integration-test goal runs our integration tests and the verify goal checks the results of our integration tests and fails the build if our integration tests failed.

After we have configured the Maven Failsafe plugin, its configuration looks as follows:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.22.1</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
        </includes>
    </configuration>
    <executions>
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

After we have configured the Maven Failsafe plugin, we have to create the Maven profiles which allow us to run either unit tests, integration tests, or all tests. Let's move on and find out how we can create these Maven profiles.

Creating the Required Maven Profiles

We can create the required Maven profiles by following these steps:

First, we have to specify the default values of following properties:

• The value of the skip.integration.tests property specifies if our build should skip our integration tests. Because only unit tests are run by default, the default value of this property must be true.
• The value of the skip.unit.tests property defines if our build should skip our unit tests. Again, because only unit tests are run by default, the default value of this property must be false.

After we have specified these default values, the properties section of our POM file looks as follows:

<properties>
    <kotlin.version>1.3.11</kotlin.version>
    <kotlin.compiler.incremental>true</kotlin.compiler.incremental>
    <junit-jupiter.version>5.3.2</junit-jupiter.version>

    <skip.integration.tests>true</skip.integration.tests>
    <skip.unit.tests>false</skip.unit.tests>
</properties>

Second, we have to create three Maven profiles:

• The dev profile is the default profile of our Maven build, and it runs only unit tests.
• The integration-test profile runs only integration tests.
• The all-tests profile runs both unit and integration tests.

After we have created these Maven profiles, the profiles section of our POM file looks as follows:

<profiles>
    <profile>
        <id>dev</id>
        <activation>
            <activeByDefault>true</activeByDefault>
        </activation>
    </profile>
    <profile>
        <id>integration-test</id>
        <properties>
            <skip.integration.tests>false</skip.integration.tests>
            <skip.unit.tests>true</skip.unit.tests>
        </properties>
    </profile>
    <profile>
        <id>all-tests</id>
        <properties>
            <skip.integration.tests>false</skip.integration.tests>
            <skip.unit.tests>false</skip.unit.tests>
        </properties>
    </profile>
</profiles>

Third, we have to ensure that the Maven Surefire plugin doesn't run our unit tests if the value of the skip.unit.tests property is true. After we have made this change to the configuration of the Maven Surefire plugin, its configuration looks as follows:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.1</version>
    <configuration>
        <skipTests>${skip.unit.tests}</skipTests>
    </configuration>
</plugin>

Fourth, we have to ensure that the Maven Failsafe plugin doesn't run our integration tests if the value of the skip.integration.tests property is true. After we have made this change to the configuration of the Maven Failsafe plugin, its configuration looks as follows:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.22.1</version>
    <configuration>
        <includes>
            <include>**/*Test.java</include>
        </includes>
    </configuration>
    <executions>
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
            <configuration>
                <skipTests>${skip.integration.tests}</skipTests>
            </configuration>
        </execution>
    </executions>
</plugin>

We have now created the required Maven profiles. Next, we will find out how we can filter the invoked tests by using JUnit 5 tags.

Filtering the Invoked Tests With JUnit 5 Tags

Because the Maven Surefire and Failsafe plugins use the same include pattern, we have to filter the invoked tests by using JUnit 5 tags. We can do this by following these steps:

First, we have to ensure that the Maven Surefire plugin runs only tests which have the tag: unitTest. After we have made the required changes to the configuration of the Maven Surefire plugin, it looks as follows:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.22.1</version>
    <configuration>
        <groups>unitTest</groups>
        <skipTests>${skip.unit.tests}</skipTests>
    </configuration>
</plugin>

Second, we have to ensure that the Maven Failsafe plugin runs only tests which have the tag: integrationTest. After we have made the required changes to the configuration of the Maven Failsafe plugin, it looks as follows:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.22.1</version>
    <configuration>
        <groups>integrationTest</groups>
        <includes>
            <include>**/*Test.java</include>
        </includes>
    </configuration>
    <executions>
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
            <configuration>
                <skipTests>${skip.integration.tests}</skipTests>
            </configuration>
        </execution>
    </executions>
</plugin>

Third, we have to ensure that our unit test class has the tag: unitTest. After we have added this tag to our unit test class, its source code looks as follows:

import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.DisplayName
import org.junit.jupiter.api.Tag
import org.junit.jupiter.api.Test

@Tag("unitTest")
class MessageServiceTest {

    private lateinit var service: MessageService

    @BeforeEach
    fun configureSystemUnderTest() {
        service = MessageService()
    }

    @Test
    @DisplayName("Should return the correct message")
    fun shouldReturnCorrectMessage() {
        val message = service.getMessage()
        assertThat(message).isEqualTo("Hello World!")
    }
}

Fourth, we have to write an integration test class which ensures that the system under test returns the correct message. After we have written this class, its source code looks as follows:

import org.assertj.core.api.Assertions
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.DisplayName
import org.junit.jupiter.api.Tag
import org.junit.jupiter.api.Test

@Tag("integrationTest")
class GetMessageTest {

    private lateinit var service: MessageService

    @BeforeEach
    fun configureSystemUnderTest() {
        service = MessageService()
    }

    @Test
    @DisplayName("Should return the correct message")
    fun shouldReturnCorrectMessage() {
        val message = service.getMessage()
        Assertions.assertThat(message).isEqualTo("Hello World!")
    }
}

We have now added integration test support to our Maven build. Let's move on and find out how we can run our unit and integration tests.

Running Our Tests

If we want to run only unit tests, we have to run the command: mvn clean test -P dev. When we run this command at command prompt, we see that Maven runs only our unit tests:

[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) 
@ running-kotlin-tests-with-maven ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.testwithspring.master.kotlin.MessageServiceTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.046 s - 
in com.testwithspring.master.kotlin.MessageServiceTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

If we want to run only integration tests, we have to run the command: mvn clean verify -P integration-test. When we run this command at command prompt, we see that Maven skips our unit tests and runs only our integration tests:

[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) 
@ running-kotlin-tests-with-maven ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-failsafe-plugin:2.22.1:integration-test (integration-tests) 
@ running-kotlin-tests-with-maven ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.testwithspring.master.kotlin.GetMessageTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.043 s - 
in com.testwithspring.master.kotlin.GetMessageTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:2.22.1:verify (integration-tests) 
@ running-kotlin-tests-with-maven ---

If we want to run both unit and integration tests, we have to run the command: mvn clean verify -P all-tests. When we run this command at command prompt, we see that Maven runs our unit and integration tests:

[INFO] --- maven-surefire-plugin:2.22.1:test (default-test) 
@ running-kotlin-tests-with-maven ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.testwithspring.master.kotlin.MessageServiceTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.056 s - 
in com.testwithspring.master.kotlin.MessageServiceTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] --- maven-failsafe-plugin:2.22.1:integration-test (integration-tests) 
@ running-kotlin-tests-with-maven ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.testwithspring.master.kotlin.GetMessageTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.058 s - 
in com.testwithspring.master.kotlin.GetMessageTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:2.22.1:verify (integration-tests) 
@ running-kotlin-tests-with-maven ---

We can now create a Maven build that can run unit and integration tests which use Kotlin and JUnit 5. Let's summarize what we learned from this blog post.

Summary

This blog post has taught us six things:

• We have to declare the Kotlin standard library (kotlin-stdlib) dependency in our POM file.
• We have to compile our test code by using the Kotlin Maven plugin.
• If our integration tests have resource files (such as DbUnit data sets), we have to add the resource directory to our Maven build by using the Build Helper Maven plugin.
• We can run our unit tests by using the Maven Surefire Plugin.
• We can run our integration tests by using the Maven Failsafe Plugin.
• We can filter the invoked tests by using Maven profiles and JUnit 5 tags.

Get the source code from Github.