Do you want to read better blog posts? If so, help me to improve my writing style!

Three Reasons Why We Should Not Use Inheritance In Our Tests

A stop sign and a hand

When we write automated tests (either unit or integration tests) for our application, we should notice pretty soon that

  1. Many test cases use the same configuration which creates duplicate code.
  2. Building objects used in our tests creates duplicates code.
  3. Writing assertions creates duplicate code.

The first thing that comes to mind is to eliminate the duplicate code. As we know, the Don’t repeat yourself (DRY) principle states that:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

So we get to work and remove the duplicate code by creating a base class (or classes) which configures our tests and provides useful testing utility methods to its subclasses.

Unfortunately, this is a very naive solution. Keep on reading and I will present three reasons why we should not use inheritance in our tests.

1. Inheritance Is Not the Right Tool for Reusing Code

DZone published a very good interview of Misko Hevery where he explains why inheritance is not the right tool for reusing code:

The point of inheritance is to take advantage of polymorphic behavior NOT to reuse code, and people miss that, they see inheritance as a cheap way to add behavior to a class. When I design code I like to think about options. When I inherit, I reduce my options. I am now sub-class of that class and cannot be a sub-class of something else. I have permanently fixed my construction to that of the superclass, and I am at a mercy of the super-class changing APIs. My freedom to change is fixed at compile time.

Although Misko Hevery was talking about writing testable code, I think that this rule applies to tests as well. But before I explain why I think this way, let’s take a closer look at the the definition of polymorphism:

Polymorphism is the provision of a single interface to entities of different types.

This is not why we use inheritance in our tests. We use inheritance because it is an easy way to reuse code or configuration. If we use inheritance in our tests, it means that

  • If we want to ensure that only the relevant code is visible to our test classes, we probably have to create a “complex” class hierarchy because putting everything in one superclass isn’t very “clean”. This makes our tests very hard to read.
  • Our test classes are in the mercy of their superclass(es), and any change which we make to a such superclass can effect its every subclass. This makes our tests “hard” to write and maintain.

So, why does this matter? It matters because tests are code too! That is why this rule applies to test code as well.

By the way, did you know that the decision to use inheritance in our tests has practical consequences as well?

2. Inheritance Can Have a Negative Effect to the Performance of Our Test Suite

If we use inheritance in our tests, it can have a negative effect to the performance of our test suite. In order to understand the reason for this, we must understand how JUnit deals with class hierarchies:

  1. Before JUnit invokes the tests of a test class, it looks for methods which are annotated with the @BeforeClass annotation. It traverses the whole class hierarchy by using reflection. After it has reached to java.lang.Object, it invokes all methods annotated with the @BeforeClass annotation (parents first).
  2. Before JUnit invokes a method which annotated with the @Test annotation, it does the same thing for methods which are annotated with the @Before annotation.
  3. After JUnit has executed the test, it looks for method which are annotated with the @After annotation, and invokes all found methods.
  4. After all tests of a test class are executed, JUnit traverses the class hierarchy again and looks for methods annotated with the @AfterClass annotation (and invokes those methods).

In other words, we are wasting CPU time in two ways:

  1. The traversal of the test class hierarchy is wasted CPU time.
  2. Invoking the setup and teardown methods is wasted CPU time if our tests don’t need them.
I learned this from a book titled Effective Unit Testing.

You might of course argue that this isn’t a big problem because it takes only a few milliseconds per test case. However, the odds are that you haven’t measured how long it really takes.

Or have you?

For example, if this takes only 2 milliseconds per test case, and our test suite has 3000 tests, our test suite is 6 seconds slower than it could be. That might not sound like a long time but it feels like eternity when we run our tests in our own computer.

It is in our best interest to keep our feedback loop as fast as possible, and wasting CPU time doesn’t help us to achieve that goal.

Also, the wasted CPU time is not the only thing that slows down our feedback loop. If we use inheritance in our test classes, we must pay a mental price as well.

3. Using Inheritance Makes Tests Harder to Read

The biggest benefits of automated tests are:

  • Tests document the way our code is working right now.
  • Tests ensure that our code is working correctly.

We want to make our tests easy to read because

  • If our tests are easy to read, it is easy to understand how our code works.
  • If our tests are easy to read, it is easy to find the problem if a test fails. If we cannot figure out what is wrong without using debugger, our test isn’t clear enough.

That is nice but it doesn’t really explain why using inheritance makes our tests harder to read. I will demonstrate what I meant by using a simple example.

Let’s assume that we have to write unit tests for the create() method of the TodoCrudServiceImpl class. The relevant part of the TodoCrudServiceImpl class looks as follows:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

@Service
public class TodoCrudServiceImpl implements TodoCrudService {

	private TodoRepository repository;
	
	@Autowired
	public TodoCrudService(TodoRepository repository) {
		this.repository = repository;
	}
		
	@Transactional
	@Overrides
	public Todo create(TodoDTO todo) {
		Todo added = Todo.getBuilder(todo.getTitle())
				.description(todo.getDescription())
				.build();
		return repository.save(added);
	}
	
	//Other methods are omitted.
}

When we start writing this test, we remember the DRY principle, and we decide to create a two abstract classes which ensure that we will not violate this principle. After all, we have to write other tests after we have finished this one, and it makes sense to reuse as much code as possible.

First, we create the AbstractMockitoTest class. This class ensures that all test methods found from its subclasses are invoked by the MockitoJUnitRunner. Its source code looks as follows:

import org.junit.runner.RunWith;
import org.mockito.runners.MockitoJUnitRunner;

@RunWith(MockitoJUnitRunner.class)
public abstract class AbstractMockitoTest {
}

Second, we create the AbstractTodoTest class. This class provides useful utility methods and constants for other test classes which tests methods related to todo entries. Its source code looks as follows:

import static org.junit.Assert.assertEquals;

public abstract class AbstractTodoTest extends AbstractMockitoTest {

	protected static final Long ID = 1L;
	protected static final String DESCRIPTION = "description";
	protected static final String TITLE = "title";

	protected TodoDTO createDTO(String title, String description) {
		retun createDTO(null, title, description);
	}

	protected TodoDTO createDTO(Long id, 
								String title, 
								String description) {
		TodoDTO dto = new DTO();
		
		dto.setId(id);
		dto.setTitle(title);
		dto.setDescrption(description);
	
		return dto;
	}
	
	protected void assertTodo(Todo actual, 
							Long expectedId, 
							String expectedTitle, 
							String expectedDescription) {
		assertEquals(expectedId, actual.getId());
		assertEquals(expectedTitle, actual.getTitle());
		assertEquals(expectedDescription, actual.getDescription());
	}
}

Now we can write a unit test for the create() method of the TodoCrudServiceImpl class. The source code of our test class looks as follows:

import org.junit.Before;
import org.junit.Test;
import org.mockito.Mock;

import static org.mockito.Matchers.isA;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.mockito.Mockito.when;

public TodoCrudServiceImplTest extends AbstractTodoTest {

	@Mock
	private TodoRepository repositoryMock;
	
	private TodoCrudServiceImpl service;
	
	@Before
	public void setUp() {
		service = new TodoCrudServiceImpl(repository);
	}
	
	@Test
	public void create_ShouldCreateNewTodoEntryAndReturnCreatedEntry() {
		TodoDTO dto = createDTO(TITLE, DESCRIPTION);
		
		when(repositoryMock.save(isA(Todo.class))).thenAnswer(new Answer<Todo>() {
	    	@Override
			public Todo answer(InvocationOnMock invocationOnMock) throws Throwable {
				Todo todo = (Todo) invocationOnMock.getArguments()[0];
				todo.setId(ID);
				return site;
			}
		});
				
		Todo created = service.create(dto);
		
		verify(repositoryMock, times(1)).save(isA(Todo.class));
		verifyNoMoreInteractions(repositoryMock);
				
		assertTodo(created, ID, TITLE, DESCRIPTION);
	}
}

Is our unit test REALLY easy read? The weirdest thing is that if we take only a quick look at it, it looks pretty clean. However, when we take a closer look at it, we start asking the following questions:

  • It seems that the TodoRepository is a mock object. This test must use the MockitoJUnitRunner. Where the test runner is configured?
  • The unit test creates new TodoDTO objects by calling the createDTO() method. Where can we find this method?
  • The unit test found from this class uses constants. Where these constants are declared?
  • The unit test asserts the information of the returned Todo object by calling the assertTodo() method. Where can we find this method?

These might seem like “small” problems. Nevertheless, finding out the answers to these questions takes time because we have to read the source code of the AbstractTodoTest and AbstractMockitoTest classes.

If we cannot understand a simple unit like this one by reading its source code, it is pretty clear that trying to understand more complex test cases is going to be very painful.

A bigger problem is that code like this makes our feedback loop a lot longer than necessary.

What Should We Do?

We just learned three reasons why we should not use inheritance in our tests. The obvious question is:

If we shouldn’t use inheritance for reusing code and configuration, what should we do?

That is a very good question, and I will answer to it in a different blog post.

If you want to learn more about writing clean tests, read all parts of my Writing Clean Tests tutorial.

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He is specialized in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

11 comments… add one

  • Hi Petri,

    I’m really looking forward to the next article. Currently I refactor my integration tests to not to use inheritance. It was possible to move some code into utility classes but when it comes to the Spring MockMVC configuration I have no idea how to refactor this without duplicating much code.

    I thought about some custom annotations, maybe you have a better idea/solution?

    Regards,
    Patrick

    Reply
    • Hi Patrick,

      If you want to follow the DRY principle, you have to keep the configuration in the base class(es) because otherwise you end up with duplicated code. I put the configuration to the test class because this way I can see everything I need in one place. This violates the DRY principle, but I am willing to do that because it makes my tests much easier to understand.

      By the way, I found a very confusing feature from JUnit when I wrote this blog post. Like I mentioned in the blog post, you can configure the used test runner in your base class:

      
      @RunWith(MockitoJUnitRunner.class)
      public abstract class AbstractMockitoTest {
      }
      
      

      Now, if you need to write tests for the TodoController class by using Spring MVC Test, you need run your tests by using the SpringJUnit4ClassRunner. Originally I thought that it is impossible to configure more than one test runner in your inheritance hierarchy. However, you can actually do it:

      
      @RunWith(SpringJUnit4ClassRunner)
      public class TodoControllerTest extends AbstractTodoTest {
      }
      
      

      After you done this, the tests found from the TodoControllerTest class are invoked by the SpringJUnit4ClassRunner.

      Isn’t it amazing (and confusing)?

      Reply
      • I never had that problem because I dislike mocking objects via the @Mock annotation and always use the SpringJUnit4ClassRunner (for integration tests). In my opinion Mocks should be initialized via the @Before annotated method or directly in the test method.

        Using the @Mock annotation isn’t intuitive, because it doesn’t indicate whether the Mock will be created just once (for all test methods in the class) or whether there will be a new instance for every test method (when I think about that I really don’t know how @Mock behaves :D).

        Reply
        • I agree that the @Mock annotation adds additional magic to the test suite. Again, this is trade-off between not writing the mock initialization code and writing the mock initialization code.

          I use it extensively in my service tests because I think that it the cleanest way to create mock objects as long as you can use the MockitoJUnitRunner class (and you know how the @Mock annotation works).

          On other hand, there are good reasons to create mock objects manually as well. You mentioned one of them in your comment.

          In the end everyone has to choose their own poison.

          Reply
  • Excellent post Petri. Your example of 3000 unit tests with 6s difference is not even close to the horrors that can be caused by inheritance in test classes. The projects I work on heavily use SpringJunit4ClassRunner and @ContextConfiguration annotation. With inheritance and @ContextConfiguration placed in the base class one soon ends up with a need for @DirtiesContext annotation in the actual test classes. Sprinkle some cargo cult coding on top of that, and what you’ll get is classMode=AFTER_EACH_TEST_METHOD. Now that’s a nice way to slow down the build :)

    Reply
    • Antti,

      Thanks! I agree that my example was a bit over-optimistic. Like you said, there are much hairier situations out there (I assume that yours is pretty hairy).

      The thing is that if the build takes unreasonable long, people don’t run tests locally before they commit their code. This means that the feedback loop is a lot longer than necessary and your build is broken more often than it should be. It all adds up…

      Reply
  • Hi Petri,

    I think your arguments do not justify the generality of your conclusion.

    1. is of course totally true. You should only inherit if there is a clear is-a relationship – being able to reuse code just usually comes as an added benefit.

    I think 2. is more controversial. First of all, the one proposing the optimization has to prove that it is worth it – not the one who is using his or her original code. So “Or have you?” right back at you. ;) I also think this is premature optimization. If your test classes are designed well (and this explicitly includes readability) and runtime doesn’t hurt you, leave it as it is.

    3. makes a good point but I think it’s not quite the general one you wanna make. And this is because you’re using a typical example of “bad inheritence”: ‘TodoCrudServiceImplTest’ is no ‘AbstractTodoTest’, so it shouldn’t inherit from it. Hence this point only counts if you weaken you’re argument to “Why We Should Not Use Bad Inheritance In Our Tests” which is clearly a subpoint of “Why We Should Not Use Bad Inheritance (Ever!)”.

    So I agree that we should not use inheritance like this in tests. But I think it is totally fine to have classes ‘Developer extends Employee’ and ‘DeveloperTest extends EmployeeTest’. Doing it this way even enforces the LSP by executing the same tests on the ‘Employee’-part of all ‘Employee’-subclasses – something which is not guaranteed if you implement separate tests for each subclass.
    Similarly, an abstract class which tests against an interface, where test implementations only have to code a bare minimum to have the full battery of tests run against their interface implementation (e.g. [1]), is nice. But then you might get into trouble when a class implements several interfaces but that can be dealt with.

    so long … Nicolai

    [1] http://code.google.com/p/google-collections/source/browse/trunk/testfw/com/google/common/collect/testing/MapInterfaceTest.java

    Reply
    • Hi Nicolai,

      I think 2. is more controversial. First of all, the one proposing the optimization has to prove that it is worth it – not the one who is using his or her original code. So “Or have you?” right back at you. ;) I also think this is premature optimization. If your test classes are designed well (and this explicitly includes readability) and runtime doesn’t hurt you, leave it as it is.

      Here is one example how inheritance can have a huge negative effect to the performance of your test suite. However, I agree that you shouldn’t start rewriting your existing tests just because you read a blog post which states that using inheritance in your test suite is a bad practice. If you can live with your existing test suite, you should not do anything to it!

      On the other hand, if you start a greenfield project, it is kind of hard to see why you would want to use inheritance in your test suite.

      3. makes a good point but I think it’s not quite the general one you wanna make. And this is because you’re using a typical example of “bad inheritence”: ‘TodoCrudServiceImplTest’ is no ‘AbstractTodoTest’, so it shouldn’t inherit from it. Hence this point only counts if you weaken you’re argument to “Why We Should Not Use Bad Inheritance In Our Tests” which is clearly a subpoint of “Why We Should Not Use Bad Inheritance (Ever!)”.

      Of course it is bad inheritance but I don’t see how it weakens my argument. This is the way inheritance is used if its only purpose is to reuse code or configuration (and this is its only purpose when it is used in a test suite).

      So I agree that we should not use inheritance like this in tests. But I think it is totally fine to have classes ‘Developer extends Employee’ and ‘DeveloperTest extends EmployeeTest’. Doing it this way even enforces the LSP by executing the same tests on the ‘Employee’-part of all ‘Employee’-subclasses – something which is not guaranteed if you implement separate tests for each subclass.

      I think that this is an interesting argument. It is a shame that it is a bit hard to evaluate it because you didn’t explain the relationship and structure of the Developer and Employee classes. I am wondering about the following things:

      • Is the Employee an abstract class?
      • Does the Developer class override the methods of the Employee class or modify the state of its Employee “part”? If so, what methods does it override and how does it modify the state of its Employee “part”?
      Reply
  • Hi Petri,

    We are in middle of setting up our framework and realised a possible bottleneck. We have a controllertest junit based class which calls different functional Libraries (Classes) through reflection as I am picking my testcases from excel. Now since these are two independent class so running through Junit / Jenkins it only report @Test of ControllerTest class. However @test method in functional lib are not reported.
    I tried using inheritance however i need to extend controller test to function lib so that @Test method can be invoked however i am not sure how many Function Libraries I will have in future.

    Please suggest a workaround to achieve the same.

    Regards,
    Manish

    Reply
    • Hi,

      Suppose my function lib class extends controllertest class. how i can report @test methods of inherited class ( functionlib class) when they will be called in controllertest class @Test methods

      Request you to please assist regarding the same

      Regards,
      Manish

      Reply
      • Hi,

        You said that Jenkins reports only the @Test methods of ControllerTest class. Did you mean that the test report / statistics created by Jenkins doesn’t contain information about the @Test methods found from the function lib class?

        Reply

Leave a Comment