12 Tools That I Use for Writing Unit and Integration Tests

I think that software development is more than just a job. I see myself as an artisan who is trying to get better every day. The "easiest way" to do this is to find a few good tools and find the answers to these questions:

  • When should I use the tool X?
  • How should I use the tool X?

Automated testing is a very important part of software development, but I haven’t seen a lot of blog posts that let you to take a peek into the toolbox of another developers.

This blog post lets you to take a peek into my toolbox. I will reveal 12 tools that I use for writing unit and integration tests. I will also provide links to other interesting webpages that help you to understand how you can use them.

Let's get started.

Taking a Peek Into My Toolbox

Before we can use the tools that are described in this blog post, we have to create a build that can run both unit and integration tests. I have written two blog posts that describe how we can do this:

We are now ready to take closer look at my toolbox. I have divided my tools into different categories that makes this blog post easier to read.

I give you 12 tools that I use for writing unit and integration tests:

Running Tests

JUnit is a framework that I use for writing both unit and integration tests. I like JUnit because it is the most popular testing framework for Java programming language. In other words, it has a lot extensions and it is easy to find solutions to your problems.

NestedRunner is a JUnit runner that allows us to run test methods placed in nested inner classes. I like NestedRunner because of the following reasons:

  • We can replace long method names with BDD style class hierarchy.
  • We can remove duplicate code by moving that code to setup methods and putting these methods to the correct inner classes.
  • We can link the constants with the test cases that use them by declaring the constants in the correct inner class.

junit-dataprovider is a JUnit runner that allows us to write parametrized tests by using a TestNG like data provider. This is a huge improvement over the default way of writing parametrized tests that pretty much sucks.

Additional Reading:

Mocking, Stubbing, and Faking Stuff

Mockito is the most popular mocking framework for writing unit tests. I like it because it has a simple API, a lot of useful features, and excellent documentation.

Greenmail is an in-memory email server that supports SMTP, POP3, and IMAP with SSL socket support. I like it because it is really easy to use. When I was looking for a "fake" email server, I tried several alternatives and Greenmail was the only one that worked as I wanted.

MockFtpServer is a library that provides two different FTP server implementations (fake/stub) that are useful for testing different scenarios. If we need test code that interacts with a FTP server, MockFtpServer is our weapon of choice.

Additional Reading:

Writing Assertions

Hamcrest provides matchers that we can use to write assertions for our unit and integration tests. I use it when I need to write assertions for unit or integration tests that use the Spring MVC Test framework.

AssertJ provides a fluent API for writing assertions that have helpful error messages. It improves the readability of our test code and helps us to transform our test cases into executable specifications that speak the correct domain-specific language.

Additional Reading:

Testing Data Access Code

H2 is a very fast in-memory database that is useful for writing integration tests that are run in the local development environment.

DbUnit is a JUnit extension that can be used to initialize the database into a known state before each integration test and ensure that the database contains the correct data. DbUnit has its warts, but it is a very useful tool because it helps us to separate the test data creation from the tested code.

Additional Reading:

  • DbUnit Core Components describes the core components of DbUnit. If we want to write tests that use DbUnit, we should know what these components are.
  • DbUnit Best Practices provides five rules that help us to write better tests for our data access code.
  • Writing Tests for Data Access Code helps us to write deterministic data access tests which test the right thing, are easy to read, and easy to maintain.

Testing Spring Applications

Spring Test is the Swiss army knife of writing automated tests to Spring applications. It provides a first class support for writing unit and integration tests to Spring powered applications.

Spring Test DbUnit integrates DbUnit with the Spring Test Framework. If we need to write data access tests for a Spring application that uses relational database, Spring Test DbUnit helps us to do it.

Additional Reading:

If you want to learn how I use these tools for writing unit, integration, and end-to-end tests for Spring web applications, take a look at my upcoming Test With Spring Course.

What Testing Tools Do You Use?

I have now revealed the tools that I use for writing unit and integration tests. I challenge you to do the same. You can participate in this challenge by following these simple steps:

  • If you have a blog, you can write a blog post and leave a comment to this blog post.
  • If you don’t have a blog, you can leave a comment to this blog post.

Why should you do this?

Well, if helping other people isn’t enough for you, I promise to help you to get more readers by linking to your blog post, tweeting it, and sharing it with the Java Testing Society.

33 comments… add one
  • Tomasz N. Jun 7, 2015 @ 21:07

    I guess you tried Spock? It replaces half of the tools you suggest: JUnit, NestedRunner, junit-dataprovider, Mockito, hamcrest, AssertJ. In order to run Spring integration tests you need an extra spock-spring plugin.

    • Petri Jun 7, 2015 @ 21:27

      I haven't tried the Spock Framework yet. It is on my to-do list and I hope that I can try it during the summer.

      • lt data Jun 8, 2015 @ 10:09

        indeed, you should try spock and you will remove half of the tools you mentioned. plus, your tests will become more readable and easier to maintain.

      • Zach Lysobey Jun 8, 2015 @ 21:02

        Yes. Spock all the way! +Geb for Browser automation tests

    • ph_maes Jun 8, 2015 @ 18:58

      First, thank you for the post. As usual, what you write is useful and interesting.

      For the section "Test Access Code" you have the well known DbUnit (http://dbunit.sourceforge.net/). It's very handy because you can easily put your database in a desired state. Moreover, it comes with several useful methods and features when you need to make tests with a database.

      More generally, you have Unitils (http://unitils.org/summary.html) that provides a huge amount of modules and features to make your life easier when you write unit & integration tests!

      • Petri Jun 9, 2015 @ 18:43

        Thank you for your kind words. I really appreciate them. Also, thank you for sharing your testing tools.

  • Gregor Riegler Jun 8, 2015 @ 18:35

    verify your exceptions in a better way:
    https://code.google.com/p/catch-exception/

    fest, fluent assertions
    https://code.google.com/p/fest/ (will be replaced by assertj probably)

    when you need a hammer
    https://code.google.com/p/catch-exception/

    testing equals is boring
    https://github.com/jqno/equalsverifier

    junit sucks at parameterising
    https://github.com/Pragmatists/junitparams

    these are the ones which came to mind right now, but there is more.

  • g00glen00b Jun 9, 2015 @ 0:35

    I'm using most of the tools mentioned in the article.
    JUnit/AssertJ/Mockito for unit tests,
    Spring test/Spring DBUnit test/DBUnit for repository integration tests
    Spring test/RESTAssured for REST API integration tests

    • Petri Jun 9, 2015 @ 18:48

      Thank you for sharing your testing tools. I left REST-assured out from my list because I haven't used it personally. However, I have heard that it is a very useful library if you need to write "real" integration tests for your REST API.

  • Arho Huttunen Jun 9, 2015 @ 8:50

    NestedRunner was new to me and I find it very interesting as you can build a structured setup hierarchy for your tests. Definitely something I'm going to try next.

    There are a lot of good tools mentioned already in the comment section but I have to mention a couple more: JsonUunit with fluent assertions for comparing JSON in unit tests. It provides neat things like treating nulls as absent and ignoring extra fields.
    https://github.com/lukas-krecan/JsonUnit

    In addition to comparing JSON it can be useful to compare XML with xmlunit.
    https://github.com/xmlunit/xmlunit

    One more that I did not see mentioned is tempus-fugit for testing concurrent code.
    http://tempusfugitlibrary.org/

    • Petri Jun 9, 2015 @ 18:45

      Hi Arto,

      Thank you for mentioning these testing libraries. I had heard about xmlunit, but JsonUnit and tempus-fugit are new to me.

  • Martin Jun 10, 2015 @ 13:24

    Hi,

    thanks for the comprehensive list!

    For people working with MongoDB I can recommend Fongo - https://github.com/fakemongo/fongo - which fakes a MongoDB (hence the name) and intercepts calls to the standard Java driver, such that it is really easy to use with frameworks like Spring Data. It provides some useful test annotations for backing your test with JSON files simulating Mongo collection contents.

    • Petri Jun 11, 2015 @ 19:43

      Hi Martin,

      Thank you for your kind words. I really appreciate them.

      Also, thank you for recommending Fongo. I will use it when I write integration tests for the example application of my blog post titled: Creating a REST API With Spring Boot and MongoDB.

  • Pedro Jun 12, 2015 @ 2:44

    javOSize, although it is not specifically designed QA tool, it is really helpful for troubleshooting and performance testing. Some Accenture guys are already using it, as I have just seen in their webpage, and it is totally free.

    As the tool seems really new I am posting the link so you can find it easily: http://www.javosize.com

    • Petri Jun 19, 2015 @ 10:05

      Hi Pedro,

      Thank you for sharing this! It looks really interesting (and a bit similar than NewRelic). I took quick look at the getting started documentation and it explains some of the most basic use cases pretty well. I will definitely evaluate this tool after my summer holiday.

  • whaley Jun 18, 2015 @ 18:27

    I've gotten some decent mileage out of awaitility (https://github.com/jayway/awaitility) when I need to test something involving asynchronous processing. It ends up being a more succinct and efficient way to poll for a result when testing an asynchronous process without littering your test methods with while loops and Thread.sleep calls.

    • Petri Jun 19, 2015 @ 9:51

      Wow. I took a quick look at the user guide and it seems that Awaitility makes tests a lot easier to read. Thank you for sharing this (I hadn't heard about it before your comment).

  • Mark Perry Jun 23, 2015 @ 16:31

    I don't get why we don't get more focus on property based testing like in Haskell. In Java, Functional Java provides the best and most faithful implementation, http://functionaljava.org/. The original paper is available at http://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/quick.pdf.

    • Petri Jun 23, 2015 @ 22:43

      Hi Mark,

      I took a very quick look at the paper and noticed that it argues that a testing tool should be able to generate test cases automatically. There is a library called RandomizedTesting that helps you to do this (if you use JUnit). Since I didn't read the entire paper yet, I won't comment property based testing, but I can share some of the challenges which I face on my work.

      My biggest challenge is to convince people to write tests. It seems that developers can be divided into four groups:

      • They don't write tests because they think automated testing is a waste of time.
      • They think that they should write tests, but they don't do it because "they don't have to do it".
      • They write automated tests because they "have to do it".
      • They are passionate about automated testing.

      Because I am passionate about automated testing, I want to help developers to either start writing them or learn new trick that help them to write better tests. Surprisingly, I spend most of my time trying to convince people to write automated tests or write tests that are easy to read and maintain. That is why I feel that it doesn't make sense to convince people to use "advanced" testing techniques when they aren't necessarily using the obvious ones yet.

  • Georgy Ryabov Jul 18, 2016 @ 16:30

    Hi.

    I switched to use TestNG (supported by Spring test) instead of JUnit.

    DbSetup - http://dbsetup.ninja-squad.com/index.html
    looks promised as replacement of DbUnit as it is Java configurable. You have not to deal with xml/yaml db descriptions.

    To write build in-container tests in Java EE world I used http://arquillian.org/ tool

  • Mike Nov 4, 2016 @ 20:57

    Hi Arho,

    Saw your posts on an interesting thread about whether one should test private methods, here: https://henrikwarne.com/2014/02/09/unit-testing-private-methods/

    I am a newbie at TDD and trying to teach myself... this is pretty hard. I have worked through Growing Object-Oriented Software Guided by Tests, and also Practical Unit Testing with JUnit and Mockito. I am getting... um, somewhere. But when I start to try to apply some of the thinking to my own projects I quickly get confused ... and find that I'm exposing too much of the SUT classes... and probably making my tests far too fine-grained.

    Your line in the above thread gives the impression that you have reached the "Holy Grail" of making the right methods public, and successfully then refactoring so that private methods emerge (effortlessly and elegantly).

    The question is, how do I get from here to there? Please could you recommend maybe another book or two, or a site or two?

    PS I see you have one book involving the Spring Framework. This is (currently) one framework too far for me. Mockito and AssertJ are my ideal combination.

    Thanks,
    Mike
    (you could just email me if you want...)

  • Muttana Mar 19, 2017 @ 22:22

    I need to test Vaadin Framework application ,which kind of tool is good other then "TestBench"

    • Petri Mar 22, 2017 @ 13:53

      Hi,

      Unfortunately I cannot help you since I have no experience from Vaadin.

  • Kapil Aug 6, 2018 @ 20:47

    Good list of tools here. My team works on Spring applications and mostly work with JUnit, Mockito, and Powermock. We leverage a tool from Parasoft called the Jtest Unit Test Assistant to create our unit test and it does it good job of actually creating readable unit tests. http://www.parasoft.com/products/jtest#unittesting

  • Serena Gray Jun 29, 2021 @ 13:12

    Found it helpful. Thanks for sharing!

Leave a Reply