The Best Way to Configure the Spring MVC Test Framework, Part Two

Before we can write integration tests for Spring MVC controllers, we have to configure the system under test (aka the Spring MVC Test framework), and that's why we must understand what is the best way to configure our integration tests.

After we have finished this blog post, we:

• Can identify the techniques we can use when we want to configure the Spring MVC Test framework.
• Can choose the best way to configure the Spring MVC Test framework when we are writing integration tests.

Let's begin.

What Options Do We Have?

When we configure the Spring MVC Test framework, we have to create a new MockMvc object which allows us to send HTTP requests to the system under test. We can create this object by using the static factory methods of the MockMvcBuilders class. When we use the MockMvcBuilders class, we can configure the Spring MVC Test framework by using one of these two options:

• The web application context based configuration loads the Spring application context by using the specified Java configuration classes or XML configuration files, and configures the system under test by using the loaded application context.
• The standalone configuration allows us to configure the Spring MVC infrastructure programmatically. This option provides the minimum configuration that allows the DispatcherServlet to serve HTTP requests which are processed by Spring MVC controllers. We can naturally customize this configuration by using a fluent API.

Next, we will take a look at the characteristics of good integration tests.

The Characteristics of Good Integration tests

Before we can identify the best way to configure the Spring MVC Test framework, we have to list our requirements and select the configuration option which fulfills these requirements. When we are writing integration tests, we can identify our requirements by identifying the characteristics of good integration tests.

A good integration test is:

Repeatable. A good integration test is deterministic. This means that if we run the same test multiple times (without making any changes to the system under test, invoked test case, or the test data), the outcome of the invoked test must be the same. We must write deterministic tests because tests which fail randomly create two problems:

• Developers lose trust in our test suite and treat every test failure as a false positive. In other words, because developers don't bother to investigate test failures, we will deploy bugs to our production environment.
• Debugging and fixing flaky tests is hard because we cannot reproduce the problem every time when we run them.

Independent. Our integration tests are independent if the outcome of the previous integration test has no effect to outcome of the next integration test AND the execution order of our integration tests has no effect to the outcome of these tests. If our integration tests aren't independent, we will face one of these two problems:

First, if we run our integration tests in random order, our test suite contains flaky tests which aren't trusted by anyone and are hard to fix. As I mentioned earlier, if we end up in this situation, we will deploy bugs to our production environment.

Second, if we run our integration tests in a specific order, we end up with tests which:

• Are hard to maintain. If we make changes to an integration test, we must ensure that our changes won't break the tests which are invoked after the updated integration test. This is slow and extremely frustrating.
• Are hard to write. Adding new tests to our test suite is hard because we have to verify that our new tests won't cause problems for the existing tests found from our test suite.
• Are hard to debug. It's hard to pinpoint the root cause of a test failure because the configuration of the failed test case is scattered into the test cases which were run before the failed test case.

As fast as possible. Integration tests can never be as fast as unit tests, but they should be as fast as possible because we want that our feedback loop is as short as possible. If our integration test suite is "too slow", we cannot expect that developers run integration tests in their development environment. Instead, we have to either run our integration tests in the CI server when a new PR is created (the best case scenario) or run our integration tests once a day (the worst case scenario).

The problem is that both of these options increase the length of our feedback loop. After a developer has created a PR, they pick a new ticket and start implementing it. If the tests of the PR fail, the developer has to stop working on their new ticket and start investigating what went wrong. This causes a context switch that's both slow and stressful. That's why we should have an integration test suite that can be run before the PR is created.

Informative. A good integration test increases our confidence that our application is working as expected when it's deployed to the production environment. That's why our integration tests should use the "same" configuration that's used in the production environment. To be more specific, our integration tests should:

• Be run against the database that's used in the production environment. If we want to verify that our database migration scripts are working in the production environment and find problems which might occur in the production environment, we must use the same database.
• Minimize the number of test doubles. We use test doubles because we cannot or don't want to use the real thing. This is a valid technique when we are writing unit tests. However, if we want to ensure that our application is working as expected when we deploy it to the production environment, our integration tests shouldn't use test doubles (unless it's absolutely necessary).

We have now identified the requirements of our integration tests. Let's move on and find out what's the best way to configure the Spring MVC Test framework when we are writing integration tests.

Choosing the Best Way to Configure the Spring MVC Test Framework

I argue that if we want to write integration tests which fulfill our requirements, we have to configure our integration tests by using the web application context based configuration. The web application context based configuration has the following benefits over the standalone (aka programmatic) configuration:

First, web application context based configuration provides an easy way to use the "same" configuration that's used in the production environment. Spring and Spring Boot web applications are quite complex beasts and it simply isn't feasible to use the programmatic configuration because of these three reasons:

• It's hard to write the code that configures our integration tests. For example, we must ensure that things like authentication, authorization, transaction management, and database connection pooling (and many other things) work in the same way as they do in the production environment. This is a slow, frustrating, and quite challenging task. In other word, it simply takes too long to write the code that configures our integration tests.
• Because it's quite challenging to configure our integration tests by using the standalone configuration, it's likely that our configuration code contains bugs which can cause false positives. This is a big problem because developers will lose trust to our test suite and we end up deploying bugs to our production environment. To make matters worse, it's not (necessarily) easy to find and fix these bugs.
• This is be a maintenance nightmare. If we configure our integration tests by using the programmatic configuration, we must ensure that our configuration code is always up-to-date. In other words, when we update our Spring or Spring Boot version, we must verify that our integration tests and our application use the same configuration. This takes a lot of work. Also, there is a possibility that we cannot update our Spring or Spring Boot version because the update contains breaking changes which would break our integration tests.

Second, the Spring TestContext framework has a feature called context caching which reduces the time that's required to load the configuration of our integration tests when we configure our integration tests by using the web application context based configuration and multiple test use the "same context configuration". In other words, if we use the web application context based configuration, it's easier to write fast integration tests because we don't have to write the code that caches the configuration of our integration tests.

Third, because we want to write deterministic and independent integration tests, we must initialize our database into a known state before an integration test is run and this is a lot easier to do if we use the web application context based configuration. For example, we can insert our test data into the database by using SQL scripts or integrate the Spring TestContext framework with DbUnit.

If we would configure our integration test by using the programmatic configuration, we would have to write the code that initializes our database into a known state before an integration test is run. This is error prone and makes our configuration more complex than it should be.

I have now explained why I think that we should configure our integration tests by using the web application context based configuration. Let's move on and talk a little bit about test doubles.

Test Doubles Aren't All Bad

Earlier I sad that:

If we want to ensure that our application is working as expected when we deploy it to the production environment, our integration tests shouldn't use test doubles (unless it's absolutely necessary).

Well, it turns out there are situations when using test doubles is absolutely necessary. These situations are:

• If we are writing integration tests for code that deals with date and time, we should use a test double that allows us to set the current date and time.
• If we are writing integration tests for code that generates random values (like UUIDs) and we want to write assertions for the generated values, we should use a test double which allows us to configure the generated value.
• If we are writing integration tests for code that uses an external HTTP API or publishes custom events, we can replace the external HTTP API with a test double (check out WireMock) or use a test double that captures the published events.

We can now select the best way to configure the Spring MVC Test framework when we are writing integration tests. Let's summarize what we learned from this blog post.

Summary

This blog post has taught us six things:

• We can configure the Spring MVC Test framework by using the standalone configuration or the web application context based configuration.
• We should configure our integration tests by using the web application context based configuration.
• Web application context based configuration provides an easy way to use the "same" configuration that's used in the production environment.
• Web application context based configuration reduces the time that's required to load the configuration of our integration tests.
• If we use the web application context based configuration, it's easy to initialize our database into a known state before an integration test is run.
• Our integration tests shouldn't use test doubles (unless it's absolutely necessary).