Do We Need Unit Tests Anymore?

The year 2024 is coming to an end and I want to celebrate the new year by writing a blog post that recollects my own "testing journey" and gives some advice for the future. After we have finished this blog post, we:

• Know how the testing tools have evolved during my software development career.
• Can identify the reasons why integration, API, and end-to-end tests have become more popular.
• Understand how we can identify the tests which are most useful to us.

Let's begin.

A Walk Down the Memory Lane — How Automated Testing Caught Fire

I have been working in software development industry for over 20 years, and when I reflect on my journey, I can see how we arrived at the situation where we are today. Although my experiences are centered in Finland, and the timeline may not be exact, I think that these memories offer some insights into how automated testing became a first class citizen, and where it could be headed.

First Job: 2000

My first software development job was in 2000 and no one had heard about automated testing. That's why we tested everything manually.

First Encounter with Automated Testing: 2003

Around 2003, I heard about automated testing for the first time. The testing tools were primitive, and finding good information about them (such as documentation and tutorials) was frustratingly difficult. The lack of resources meant that we didn’t write many tests. We did some experiments, but we noticed almost immediately that the testing tools had so many limitations (or the documentation was non-existent) that they weren't useful to us.

Renewed Interest: 2006

In 2006, automated testing came up again as few developers began to experiment with unit tests. The majority, however, saw little value in it and didn't write any tests. Resources were still pretty rare and that's why it was challenging to explore testing tools. We wrote primarily unit tests by using whatever libraries we could find, but these tools had limitations that still echo in today’s "best practices".

Integration tests were rare because we didn't have a good way to insert test data into the database. Although the first version of DbUnit was released as early as 2002, we hadn’t discovered it yet. Still, I remember that one developer suggested that we could insert test data into the database by using JDBC API. At the time our application used Hibernate, and he argued that his approach ensured that the data is fetched from the database instead of the Hibernate session. We wrote some integration tests and noticed that his approach works, but it was labor-intensive which limited our ability to write integration tests.

Finally, managers weren't necessarily convinced that writing tests was worth it.

The Rise of "TDD": 2010-2011

By 2010-2011, "Test-Driven Development (TDD)" began gaining traction. Yet, developers were divided: a small group, including myself, wrote a lot of unit tests, but there were many developers who didn't write tests at all. At the time, our unit tests relied heavily on test doubles like mocks and stubs. Also, most of the developers who wrote unit tests didn't do real TDD. At the time TDD kind of meant that you wrote unit tests.

A Game-Changer: 2012

In 2012, I discovered spring-test-mvc. It was an unofficial library that provided a test environment which allowed us to "send HTTP requests" to Spring MVC controllers and write assertions for the "HTTP responses". For the first time, TDD concepts began to make sense — we could use familiar language for specifying requests and expected responses, and for the first time, our tests truly identified the requirements of our application.

Around this time, I also found spring-test-dbunit, a library that integrated DbUnit with the Spring Test framework. With these libraries, we crafted integration tests which ensured that the correct information was saved to the database, the correct information was returned from the database, and the authorization rules of our application were working as expected.

However, despite these advances, most of our tests were still unit tests which overused test doubles.

The Golden Age of Unit Testing: 2014-2019

By the mid-2010s, unit testing was basically a first class citizen. Most developers either embraced it or simply accepted their fate and wrote unit tests for their code. There were some people who criticized the fact that unit tests were overusing test doubles, but this critique was mostly ignored.

Spring Boot was released and its testing enhancements made integration testing easier than ever. Slowly, developers started to write integration tests, but they faced some challenges, especially when they tried to run external services which were used by the system under test.

Toward the end of the decade, we saw interesting advancements in testing tools and frameworks. These tools offered solutions to well known pain points such as setting up complex test environments and dealing with external APIs. However, despite their potential, these innovations often didn't get the attention they deserved.

A New Era Begins: 2020 and Beyond

As the new decade unfolded, testing practices shifted and the tools from the late 2010s finally started to get the attention they deserved. These advancements made it easier to write proper integration, API, and end-to-end tests because it was easy run anything that can run in a Docker container. Thus, we could finally setup complex test environments which is something we couldn't do before.

Integration testing became a first-class citizen, and often it received more focus than unit testing. As it became easier to mirror production environments, integration testing took the center stage and we said goodbye to the unit-test-heavy approach of the previous years.

Beyond Unit Testing — The Rise of Modern Testing Practices

In recent years, several trends have forced us to face the fact that writing only unit tests isn't enough. One major factor is the rise of continuous integration and agile methodologies. Because we want to deploy our application to the production environment multiple times a day or at the end of every sprint, we must have a fast and reliable way to ensure that everything is working as expected. We can no longer rely only on manual regression testing because it simply takes too much time and resources. Instead, we should put our faith in automated testing because it helps us to keep up with rapid development cycles.

The shift to modern architectural styles—such as microservices, serverless computing, and cloud-based systems—has also played a significant role. The applications which use these architectural styles consist of "small" independent components which are working together. While unit tests help us to make sure that a single component is working as expected, unit tests won't allow us to verify that these components interact correctly when they are deployed to the production environment. This has created a need for automated tests which verify that the integrations between different components are working as expected.

Fortunately, new testing tools have emerged to meet this demand. Tools like JUnit 5, Testcontainers, WireMock, and various Spring testing enhancements have revolutionized automated testing by making difficult or nearly impossible tasks much more easier. In the past, we were forced to concentrate our efforts on writing unit tests because writing other tests was so complex and time-consuming that it wasn't worth the effort. Luckily, these advances have shifted the balance and improved the ROI of integration, API, and end-to-end tests.

In addition, the automated testing community has grown significantly. Many bloggers and YouTubers now create content that fills the gaps left by official documentation. The tips and guidance provided by these content creators help developers to write their first automated tests or improve their existing test suites. The easy access to relevant information helps developers to improve their testing game and write more integration, API, and end-to-end tests.

Finally, the combination of better testing tools and accessible learning resources has helped developers to see the true value of automated testing. Writing automated tests is no longer seen as just following a best practice. Instead, an increasing number of developers understand that a comprehensive test suite reduces manual work and increases their confidence in their own code. Most importantly, they understand that investing in a comprehensive test suite pays off significantly over time.

Rethinking Automated Testing for Tomorrow

It's time to get rid of our fixation on different test types such as unit, integration, API, and end-to-end tests. Ultimately, the purpose of an automated test is to provide us information about the system under test. Thus, we shouldn't concentrate on test types. Instead, we should ask the question: "What kind of information do we need?", and write the tests which provide the required information.

It's also important to acknowledge that there is no universal approach to automated testing. Different applications, and sometimes even different modules (or features) within the same application, require different testing strategies. That's why we must abandon best practices and think what we want to achieve before we will write any tests. Moreover, we must remember that writing automated tests isn't good enough. We must also do exploratory testing because it helps us to uncover issues which cannot be detected by automated tests.

So, what about unit tests? Do we still need them or should we concentrate on writing integration, API, or end-to-end tests?

Well, it depends. The term "unit test" is somewhat problematic. For some reason, it often encourages developers to write so called class tests which overuse test doubles. These tests aren't very useful because they tend to cause more problems than they solve. For example, class tests are fragile and limit our ability to refactor our code.

A better term could be:

• A test that has no external dependencies or
• A test which doesn't use external services

If we embrace our new definition, writing unit tests is a good idea as long as unit tests provide us the answers we seek. For example, unit tests might be useful to us if any of the following conditions is met:

The way we work. For example, if we want to do TDD, we should consider writing unit tests because they provide a shorter feedback loop.

Acceptable Risk level. The lower the acceptable risk level is, the higher our test coverage must be. Generally speaking, it's easier to achieve a high test coverage if the tests are written "near" the system under test because configuring the high-level tests (integration, API, and end-to-end) requires more work. In other words, if the acceptable risk level is low, we should write more unit tests as long as they provide the information we need.

System under test. There are situations when the system under test "dictates" what kind of tests are useful to us. For example:

• If the system under test has complex authorization rules, business rules, and/or validation rules which are used to validate the input data, it's simpler and faster to write unit tests for the components which enforce these rules than to test these rules by writing only integration, API, or end-to-end tests.
• If the system under test has no external dependencies (for example, it's a library that contains different utility classes), we should test it by writing unit tests.

We can now identify the reasons why integration, API, and end-to-end tests have become more popular and we should understand how we can the identify the tests which are most useful to us. Let's summarize what we learned from this blog post.

Summary

This blog post has taught us three things:

• The "advanced" testing tools, the rise of the modern architectural styles, and the growth of the automating testing community have shifted our focus from unit tests to integration, API, end-to-tests.
• We must get rid of our fixation on different test types such as unit, integration, API, and end-to-end tests. Instead, we should write the tests which provide us the information we need.
• Although we should pay more attention on writing integration, API, or end-to-end tests, there are situations when writing unit tests is still worth it.