Three Reasons Why We Should Not Use Inheritance In Our Tests

When we write automated tests (either unit or integration tests) for our application, we should notice pretty soon that

  1. Many test cases use the same configuration which creates duplicate code.
  2. Building objects used in our tests creates duplicates code.
  3. Writing assertions creates duplicate code.

The first thing that comes to mind is to eliminate the duplicate code. As we know, the Don't repeat yourself (DRY) principle states that:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

So we get to work and remove the duplicate code by creating a base class (or classes) which configures our tests and provides useful testing utility methods to its subclasses.

Unfortunately, this is a very naive solution. Keep on reading and I will present three reasons why we should not use inheritance in our tests.

1. Inheritance Is Not the Right Tool for Reusing Code

DZone published a very good interview of Misko Hevery where he explains why inheritance is not the right tool for reusing code:

The point of inheritance is to take advantage of polymorphic behavior NOT to reuse code, and people miss that, they see inheritance as a cheap way to add behavior to a class. When I design code I like to think about options. When I inherit, I reduce my options. I am now sub-class of that class and cannot be a sub-class of something else. I have permanently fixed my construction to that of the superclass, and I am at a mercy of the super-class changing APIs. My freedom to change is fixed at compile time.

Although Misko Hevery was talking about writing testable code, I think that this rule applies to tests as well. But before I explain why I think this way, let's take a closer look at the the definition of polymorphism:

Polymorphism is the provision of a single interface to entities of different types.

This is not why we use inheritance in our tests. We use inheritance because it is an easy way to reuse code or configuration. If we use inheritance in our tests, it means that

  • If we want to ensure that only the relevant code is visible to our test classes, we probably have to create a "complex" class hierarchy because putting everything in one superclass isn't very "clean". This makes our tests very hard to read.
  • Our test classes are in the mercy of their superclass(es), and any change which we make to a such superclass can effect its every subclass. This makes our tests "hard" to write and maintain.

So, why does this matter? It matters because tests are code too! That is why this rule applies to test code as well.

By the way, did you know that the decision to use inheritance in our tests has practical consequences as well?

2. Inheritance Can Have a Negative Effect to the Performance of Our Test Suite

If we use inheritance in our tests, it can have a negative effect to the performance of our test suite. In order to understand the reason for this, we must understand how JUnit deals with class hierarchies:

  1. Before JUnit invokes the tests of a test class, it looks for methods which are annotated with the @BeforeClass annotation. It traverses the whole class hierarchy by using reflection. After it has reached to java.lang.Object, it invokes all methods annotated with the @BeforeClass annotation (parents first).
  2. Before JUnit invokes a method which annotated with the @Test annotation, it does the same thing for methods which are annotated with the @Before annotation.
  3. After JUnit has executed the test, it looks for method which are annotated with the @After annotation, and invokes all found methods.
  4. After all tests of a test class are executed, JUnit traverses the class hierarchy again and looks for methods annotated with the @AfterClass annotation (and invokes those methods).

In other words, we are wasting CPU time in two ways:

  1. The traversal of the test class hierarchy is wasted CPU time.
  2. Invoking the setup and teardown methods is wasted CPU time if our tests don't need them.
I learned this from a book titled Effective Unit Testing.

You might of course argue that this isn't a big problem because it takes only a few milliseconds per test case. However, the odds are that you haven't measured how long it really takes.

Or have you?

For example, if this takes only 2 milliseconds per test case, and our test suite has 3000 tests, our test suite is 6 seconds slower than it could be. That might not sound like a long time but it feels like eternity when we run our tests in our own computer.

It is in our best interest to keep our feedback loop as fast as possible, and wasting CPU time doesn’t help us to achieve that goal.

Also, the wasted CPU time is not the only thing that slows down our feedback loop. If we use inheritance in our test classes, we must pay a mental price as well.

3. Using Inheritance Makes Tests Harder to Read

The biggest benefits of automated tests are:

  • Tests document the way our code is working right now.
  • Tests ensure that our code is working correctly.

We want to make our tests easy to read because

  • If our tests are easy to read, it is easy to understand how our code works.
  • If our tests are easy to read, it is easy to find the problem if a test fails. If we cannot figure out what is wrong without using debugger, our test isn't clear enough.

That is nice but it doesn't really explain why using inheritance makes our tests harder to read. I will demonstrate what I meant by using a simple example.

Let's assume that we have to write unit tests for the create() method of the TodoCrudServiceImpl class. The relevant part of the TodoCrudServiceImpl class looks as follows:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class TodoCrudServiceImpl implements TodoCrudService {

    private TodoRepository repository;
    
    @Autowired
    public TodoCrudService(TodoRepository repository) {
        this.repository = repository;
    }
        
    @Transactional
    @Overrides
    public Todo create(TodoDTO todo) {
        Todo added = Todo.getBuilder(todo.getTitle())
                .description(todo.getDescription())
                .build();
        return repository.save(added);
    }
    
    //Other methods are omitted.
}

When we start writing this test, we remember the DRY principle, and we decide to create a two abstract classes which ensure that we will not violate this principle. After all, we have to write other tests after we have finished this one, and it makes sense to reuse as much code as possible.

First, we create the AbstractMockitoTest class. This class ensures that all test methods found from its subclasses are invoked by the MockitoJUnitRunner. Its source code looks as follows:

import org.junit.runner.RunWith;
import org.mockito.runners.MockitoJUnitRunner;

@RunWith(MockitoJUnitRunner.class)
public abstract class AbstractMockitoTest {
}

Second, we create the AbstractTodoTest class. This class provides useful utility methods and constants for other test classes which tests methods related to todo entries. Its source code looks as follows:

import static org.junit.Assert.assertEquals;

public abstract class AbstractTodoTest extends AbstractMockitoTest {

    protected static final Long ID = 1L;
    protected static final String DESCRIPTION = "description";
    protected static final String TITLE = "title";

    protected TodoDTO createDTO(String title, String description) {
        retun createDTO(null, title, description);
    }

    protected TodoDTO createDTO(Long id, 
                                String title, 
                                String description) {
        TodoDTO dto = new DTO();
        
        dto.setId(id);
        dto.setTitle(title);
        dto.setDescrption(description);
    
        return dto;
    }
    
    protected void assertTodo(Todo actual, 
                            Long expectedId, 
                            String expectedTitle, 
                            String expectedDescription) {
        assertEquals(expectedId, actual.getId());
        assertEquals(expectedTitle, actual.getTitle());
        assertEquals(expectedDescription, actual.getDescription());
    }
}

Now we can write a unit test for the create() method of the TodoCrudServiceImpl class. The source code of our test class looks as follows:

import org.junit.Before;
import org.junit.Test;
import org.mockito.Mock;

import static org.mockito.Matchers.isA;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.mockito.Mockito.when;

public TodoCrudServiceImplTest extends AbstractTodoTest {

    @Mock
    private TodoRepository repositoryMock;
    
    private TodoCrudServiceImpl service;
    
    @Before
    public void setUp() {
        service = new TodoCrudServiceImpl(repositoryMock);
    }
    
    @Test
    public void create_ShouldCreateNewTodoEntryAndReturnCreatedEntry() {
        TodoDTO dto = createDTO(TITLE, DESCRIPTION);
        
        when(repositoryMock.save(isA(Todo.class))).thenAnswer(new Answer<Todo>() {
            @Override
            public Todo answer(InvocationOnMock invocationOnMock) throws Throwable {
                Todo todo = (Todo) invocationOnMock.getArguments()[0];
                todo.setId(ID);
                return site;
            }
        });
                
        Todo created = service.create(dto);
        
        verify(repositoryMock, times(1)).save(isA(Todo.class));
        verifyNoMoreInteractions(repositoryMock);
                
        assertTodo(created, ID, TITLE, DESCRIPTION);
    }
}

Is our unit test REALLY easy read? The weirdest thing is that if we take only a quick look at it, it looks pretty clean. However, when we take a closer look at it, we start asking the following questions:

  • It seems that the TodoRepository is a mock object. This test must use the MockitoJUnitRunner. Where the test runner is configured?
  • The unit test creates new TodoDTO objects by calling the createDTO() method. Where can we find this method?
  • The unit test found from this class uses constants. Where these constants are declared?
  • The unit test asserts the information of the returned Todo object by calling the assertTodo() method. Where can we find this method?

These might seem like "small" problems. Nevertheless, finding out the answers to these questions takes time because we have to read the source code of the AbstractTodoTest and AbstractMockitoTest classes.

If we cannot understand a simple unit like this one by reading its source code, it is pretty clear that trying to understand more complex test cases is going to be very painful.

A bigger problem is that code like this makes our feedback loop a lot longer than necessary.

What Should We Do?

We just learned three reasons why we should not use inheritance in our tests. The obvious question is:

If we shouldn’t use inheritance for reusing code and configuration, what should we do?

That is a very good question, and I will answer to it in a different blog post.

33 comments… add one
  • Patrick Apr 20, 2014 @ 14:27

    Hi Petri,

    I'm really looking forward to the next article. Currently I refactor my integration tests to not to use inheritance. It was possible to move some code into utility classes but when it comes to the Spring MockMVC configuration I have no idea how to refactor this without duplicating much code.

    I thought about some custom annotations, maybe you have a better idea/solution?

    Regards,
    Patrick

    • Petri Apr 21, 2014 @ 11:20

      Hi Patrick,

      If you want to follow the DRY principle, you have to keep the configuration in the base class(es) because otherwise you end up with duplicated code. I put the configuration to the test class because this way I can see everything I need in one place. This violates the DRY principle, but I am willing to do that because it makes my tests much easier to understand.

      By the way, I found a very confusing feature from JUnit when I wrote this blog post. Like I mentioned in the blog post, you can configure the used test runner in your base class:

      
      @RunWith(MockitoJUnitRunner.class)
      public abstract class AbstractMockitoTest {
      }
      
      

      Now, if you need to write tests for the TodoController class by using Spring MVC Test, you need run your tests by using the SpringJUnit4ClassRunner. Originally I thought that it is impossible to configure more than one test runner in your inheritance hierarchy. However, you can actually do it:

      
      @RunWith(SpringJUnit4ClassRunner)
      public class TodoControllerTest extends AbstractTodoTest {
      }
      
      

      After you done this, the tests found from the TodoControllerTest class are invoked by the SpringJUnit4ClassRunner.

      Isn't it amazing (and confusing)?

      • Patrick May 4, 2014 @ 15:15

        I never had that problem because I dislike mocking objects via the @Mock annotation and always use the SpringJUnit4ClassRunner (for integration tests). In my opinion Mocks should be initialized via the @Before annotated method or directly in the test method.

        Using the @Mock annotation isn't intuitive, because it doesn't indicate whether the Mock will be created just once (for all test methods in the class) or whether there will be a new instance for every test method (when I think about that I really don't know how @Mock behaves :D).

        • Petri May 4, 2014 @ 15:36

          I agree that the @Mock annotation adds additional magic to the test suite. Again, this is trade-off between not writing the mock initialization code and writing the mock initialization code.

          I use it extensively in my service tests because I think that it the cleanest way to create mock objects as long as you can use the MockitoJUnitRunner class (and you know how the @Mock annotation works).

          On other hand, there are good reasons to create mock objects manually as well. You mentioned one of them in your comment.

          In the end everyone has to choose their own poison.

    • Matt F Nov 23, 2017 @ 20:03

      You said:

      "If you want to follow the DRY principle, you have to keep the configuration in the base class(es) because otherwise you end up with duplicated code. I put the configuration to the test class because this way I can see everything I need in one place. This violates the DRY principle, but I am willing to do that because it makes my tests much easier to understand."

      What's the difference between doing this in a test and doing this in a normal class? In a normal class using inheritance you could ask the same questions ("Where can we find this method?", "Where these constants are declared?") but we don't treat that as a reason to put the code in same class and repeat ourselves.

      • Petri Nov 29, 2017 @ 22:25

        Hi Matt,

        This post (and the comment in question) was written three years ago. This means that my views have changed a bit, and I don't necessarily consider inheritance to be a crime against humanity. I tend to avoid it though (more about this later).

        What’s the difference between doing this in a test and doing this in a normal class? In a normal class using inheritance you could ask the same questions (“Where can we find this method?”, “Where these constants are declared?”) but we don’t treat that as a reason to put the code in same class and repeat ourselves.

        I think that test and production code are a bit different. For example, when I write tests for my code, I typically have at least these three goals:

        • I want to find out if the design of my production goal has problems.
        • I want to ensure that my code is working as I expect.
        • I want that my tests document the behavior of my code.

        This means that if duplicate code helps me to fulfill one of these goals, I will add it to my test suite. That being said, I also think that you have to always analyze the pros and cons of this decision before you actually do it.

        In other words, if I write production code, I tend to minimize duplicate code (at least inside the same module) because it makes my code hard to maintain. However, I can tolerate it in my test code if it helps to write tests that fulfill the goals I mentioned earlier.

        Now, a general comment about inheritance: I try to use composition instead of inheritance because most of the time it makes more sense. That being said, there are situations when inheritance is a good choice. You just have to learn how to identify them. Also, if you are writing tests, you can often use testing tools that help you to configure your test cases without inheritance (like runners, rules, JUnit 5 extensions, and so on). The good thing about these tools is that they also help the reader to see immediately what's going on because he/she doesn't have to navigate to the super class.

  • Antti Koivisto Apr 21, 2014 @ 11:48

    Excellent post Petri. Your example of 3000 unit tests with 6s difference is not even close to the horrors that can be caused by inheritance in test classes. The projects I work on heavily use SpringJunit4ClassRunner and @ContextConfiguration annotation. With inheritance and @ContextConfiguration placed in the base class one soon ends up with a need for @DirtiesContext annotation in the actual test classes. Sprinkle some cargo cult coding on top of that, and what you'll get is classMode=AFTER_EACH_TEST_METHOD. Now that's a nice way to slow down the build :)

    • Petri Apr 21, 2014 @ 12:05

      Antti,

      Thanks! I agree that my example was a bit over-optimistic. Like you said, there are much hairier situations out there (I assume that yours is pretty hairy).

      The thing is that if the build takes unreasonable long, people don't run tests locally before they commit their code. This means that the feedback loop is a lot longer than necessary and your build is broken more often than it should be. It all adds up...

  • Nicolai Parlog Apr 25, 2014 @ 13:50

    Hi Petri,

    I think your arguments do not justify the generality of your conclusion.

    1. is of course totally true. You should only inherit if there is a clear is-a relationship - being able to reuse code just usually comes as an added benefit.

    I think 2. is more controversial. First of all, the one proposing the optimization has to prove that it is worth it - not the one who is using his or her original code. So "Or have you?" right back at you. ;) I also think this is premature optimization. If your test classes are designed well (and this explicitly includes readability) and runtime doesn't hurt you, leave it as it is.

    3. makes a good point but I think it's not quite the general one you wanna make. And this is because you're using a typical example of "bad inheritence": 'TodoCrudServiceImplTest' is no 'AbstractTodoTest', so it shouldn't inherit from it. Hence this point only counts if you weaken you're argument to "Why We Should Not Use Bad Inheritance In Our Tests" which is clearly a subpoint of "Why We Should Not Use Bad Inheritance (Ever!)".

    So I agree that we should not use inheritance like this in tests. But I think it is totally fine to have classes 'Developer extends Employee' and 'DeveloperTest extends EmployeeTest'. Doing it this way even enforces the LSP by executing the same tests on the 'Employee'-part of all 'Employee'-subclasses - something which is not guaranteed if you implement separate tests for each subclass.
    Similarly, an abstract class which tests against an interface, where test implementations only have to code a bare minimum to have the full battery of tests run against their interface implementation (e.g. [1]), is nice. But then you might get into trouble when a class implements several interfaces but that can be dealt with.

    so long ... Nicolai

    [1] http://code.google.com/p/google-collections/source/browse/trunk/testfw/com/google/common/collect/testing/MapInterfaceTest.java

    • Petri Apr 28, 2014 @ 21:59

      Hi Nicolai,

      I think 2. is more controversial. First of all, the one proposing the optimization has to prove that it is worth it – not the one who is using his or her original code. So “Or have you?” right back at you. ;) I also think this is premature optimization. If your test classes are designed well (and this explicitly includes readability) and runtime doesn’t hurt you, leave it as it is.

      Here is one example how inheritance can have a huge negative effect to the performance of your test suite. However, I agree that you shouldn't start rewriting your existing tests just because you read a blog post which states that using inheritance in your test suite is a bad practice. If you can live with your existing test suite, you should not do anything to it!

      On the other hand, if you start a greenfield project, it is kind of hard to see why you would want to use inheritance in your test suite.

      3. makes a good point but I think it’s not quite the general one you wanna make. And this is because you’re using a typical example of “bad inheritence”: ‘TodoCrudServiceImplTest’ is no ‘AbstractTodoTest’, so it shouldn’t inherit from it. Hence this point only counts if you weaken you’re argument to “Why We Should Not Use Bad Inheritance In Our Tests” which is clearly a subpoint of “Why We Should Not Use Bad Inheritance (Ever!)”.

      Of course it is bad inheritance but I don't see how it weakens my argument. This is the way inheritance is used if its only purpose is to reuse code or configuration (and this is its only purpose when it is used in a test suite).

      So I agree that we should not use inheritance like this in tests. But I think it is totally fine to have classes ‘Developer extends Employee’ and ‘DeveloperTest extends EmployeeTest’. Doing it this way even enforces the LSP by executing the same tests on the ‘Employee’-part of all ‘Employee’-subclasses – something which is not guaranteed if you implement separate tests for each subclass.

      I think that this is an interesting argument. It is a shame that it is a bit hard to evaluate it because you didn't explain the relationship and structure of the Developer and Employee classes. I am wondering about the following things:

      • Is the Employee an abstract class?
      • Does the Developer class override the methods of the Employee class or modify the state of its Employee "part"? If so, what methods does it override and how does it modify the state of its Employee "part"?
  • manish May 27, 2014 @ 22:11

    Hi Petri,

    We are in middle of setting up our framework and realised a possible bottleneck. We have a controllertest junit based class which calls different functional Libraries (Classes) through reflection as I am picking my testcases from excel. Now since these are two independent class so running through Junit / Jenkins it only report @Test of ControllerTest class. However @test method in functional lib are not reported.
    I tried using inheritance however i need to extend controller test to function lib so that @Test method can be invoked however i am not sure how many Function Libraries I will have in future.

    Please suggest a workaround to achieve the same.

    Regards,
    Manish

    • manish May 27, 2014 @ 22:16

      Hi,

      Suppose my function lib class extends controllertest class. how i can report @test methods of inherited class ( functionlib class) when they will be called in controllertest class @Test methods

      Request you to please assist regarding the same

      Regards,
      Manish

      • Petri May 29, 2014 @ 20:35

        Hi,

        You said that Jenkins reports only the @Test methods of ControllerTest class. Did you mean that the test report / statistics created by Jenkins doesn't contain information about the @Test methods found from the function lib class?

  • Charles Roth Mar 13, 2015 @ 17:19

    Good article. It puts into more precise words, the feelings that I've been following as the unit-test guru for our company.

    The only case where I use inheritance in our tests is to enforce a particular kind of clean-up behaviour after EVERY test. (We unfortunately have tests that call objects that start threads that don't end that eat up RAM... in the house that Jack built. So the inherited AfterClass hunts them down and kills them.) Strictly speaking, it should just be a call in every test class to do that, but I found it easier to enforce the only-one-inheritance rule, than to enforce "make sure you call this cleanup in every test class".

    Thanks again.

    • Petri Mar 14, 2015 @ 11:53

      Hi Charles,

      Thank you for your comment. It reminded me of one important "rule" that is sometimes overlooked by developers (me included). The thing is that it is a lot easier to invent "best practices" than to enforce them in real life. I think that theory is very valuable, but the ability to make the necessary tradeoffs is the difference between great and mediocre developers.

      If we blindly follow the "rules" invented by other developers, do we really learn anything? Also, aren't we ignoring the fact that rules and best practices only work in specific situations? Are we sure that the rule will help us to solve our problem? These are the questions that we should ask before we start enforcing yet another best practice.

      By the way, it seems that using inheritance makes sense in your situation because you said that:

      I found it easier to enforce the only-one-inheritance rule, than to enforce “make sure you call this cleanup in every test class”.

      If a thing X helps you to write "better" tests, you should use X even if using Y would be the correct way (theoretically speaking).

  • Russell Oct 7, 2015 @ 1:03

    If we shouldn’t use inheritance for reusing code and configuration, what should we do?

    Answer?

    • Petri Oct 8, 2015 @ 19:56

      I have described my current approach in my Writing Clean Tests tutorial. You should read all parts since they are meant to be read as a sequel and each part makes improvements to the outcome of the previous part.

      If you don't have time to read these blog posts, I will give you a summary:

      • I use nested configuration.
      • I create the required objects by using factory methods or builders because it is a much cleaner way to reuse code than inheritance.

      If you have any additional questions, don't hesitate to ask them.

  • Oscar Mar 19, 2016 @ 13:45

    Very interesting post! But I could not find the sequel ... is it perhaps posted on another site?

    • Petri Mar 19, 2016 @ 18:07

      Hi,

      It seems that I forgot to add the link to this blog post. I think that you have two options:

      • If you want to know how you can configure your unit tests without using inheritance, read this blog post.
      • If you want to read my thoughts about writing clean tests, read this tutorial.
  • Manuel Jordan Mar 8, 2018 @ 5:05

    Hi Petri

    In the `TodoCrudServiceImplTest` class exits the following line:
    `service = new TodoCrudServiceImpl(repository);`
    I think should be `service = new TodoCrudServiceImpl(repositoryMock);` instead

    Good post of course

    • Petri Mar 12, 2018 @ 10:42

      Hi,

      Thank you for pointing this mistake out! I fixed it now. Also, it was nice to hear that this blog post was useful to you.

  • Anitha Jul 14, 2018 @ 14:17

    Thanks for sharing about the article

    • Petri Jul 16, 2018 @ 1:29

      You are welcome.

  • Craig S. Aug 22, 2018 @ 23:59

    I agree. I wrote a blog article once on something called "contextual hierarchical testing", whereby inheritance was used to "encapsulate" and "de-duplicate" common test setup code. It was cool, and it worked--for THIS ONE PARTICULAR case of testing. When I tried to use it in a more generalized fashion, it just DID NOT work/scale at all. I wrote a follow-up blog saying DON'T DO THIS. You live, you learn.

  • Adam B. Oct 26, 2021 @ 17:24

    I think, "When we start writing this test, we remember the DRY principle, and we decide to create a two abstract classes which ensure that we will not violate this principle." should be, "When we start writing this test, we remember the DRY principle, and we decide to create two abstract classes which ensure that we will not violate this principle." just by removing the 'a'. Great article thanks for writing it.

    • Adam B. Oct 26, 2021 @ 17:26

      Also in the code
      protected TodoDTO createDTO(String title, String description) {
      retun createDTO(null, title, description);
      }
      return is misspelled.

      • Adam B. Oct 26, 2021 @ 17:28

        Also I believe in this part, "The unit test found from this class uses constants. Where these constants are declared?" the second sentence should read either, "Where are these constants declared?" or, "Were these constants declared?".

        I don't mean to keep commenting I'm just adding them as I read. Again thanks for writing this.

  • Alex B. Jul 4, 2023 @ 17:39

    I'd anyway use inheritance in spring-boot testing, cuz inheritance allows to create shared cached application context so tests are bootstrapped faster.
    Without inheritance each test class needs to be configured separately and this becomes a difficult task.

    • Petri Jul 4, 2023 @ 18:21

      Hi,

      Thank you for posting an interesting comment. I want to point out a few things:

      First, you can use application context caching even if you don't use inheritance because the context caching is based on so called cache keys which are generated automatically by the Spring TestContext framework.

      Second, this post was published on April 20, 2014. The world was quite different back then and we simply didn't have access to the testing tools which we have today. This changes things a bit. For example, if you want to run multiple Docker containers with TestContainers, it might be a good idea to use inheritance.

      On the other hand, if you just want to write integration tests for a REST API which uses a relational database, you don't need inheritance. All you have to do is to:

      Third, if you want to use different test data for different test cases, you still have to provide custom configuration for each test case (not the whole configuration though).

Leave a Reply