Writing Clean Tests - Naming Matters

When we are writing automated tests for our application, we have to name our test classes, our test methods, fields of our test classes, and the local variables found from our test methods.

If we want to write tests which are easy to read, we have to stop coding on autopilot and pay attention to naming.

This is easier said than done.

That is why I decided to write a blog post which identifies the problems caused by bad naming, and provides solutions to these problems.

The Devil Is in the Details

It is relatively easy to write tests which seem clean. However, if we want to go the extra mile and change our tests into a executable specification, we have to pay extra attention to the naming of test classes, test methods, test class’ fields, and local variables.

Let's find out what this means.

Naming Test Classes

When we think about the different test classes which we create in a typical project, we notice that these classes can be divided into two groups:

  • The first group contains tests which tests the methods of a single class. These tests can be either unit tests or integration tests written for our repositories.
  • The second group contains integration tests which ensure that a single feature is working properly.

A good name identifies the tested class or feature. In other words, we should name our test classes by following these rules:

  1. If the test class belongs to the first group, we should name it by using this formula: [The name of the tested class]Test. For example, if we are writing tests for the RepositoryUserService class, the name of our test class should be: RepositoryUserServiceTest. The benefit of this approach is that if a test fails, this rule helps us figure out which class is broken without reading the test code.
  2. If the class belongs to the second group, we should name it by using this formula: [The name of the tested feature]Test. For example, if we would be writing tests for the registration feature, the name of our test class should be RegistrationTest. The idea behind this rule is that if a test fails, using this naming convention helps us to figure out what feature is broken without reading the test code.

Naming Test Methods

I am big fan of the naming convention introduced by Roy Osherove. Its idea is to describe the tested method (or feature), expected input or state, and expected behavior in the name of a test method.

In other words, if we follow this naming convention, we should name our test methods as follows:

  1. If we write tests for a single class, we should name our test methods by using this formula: [the name of the tested method]_[expected input / tested state]_[expected behavior]. For example, if we write a unit test for a registerNewUserAccount() method which throws an exception when the given email address is already associated with an existing user account, we should name our test method as follows: registerNewUserAccount_ExistingEmailAddressGiven_ShouldThrowException().
  2. If we write tests for a single feature, we should name our test methods by using this formula: [the name of the tested feature]_[expected input / tested state]_[expected behavior]. For example, if we write an integration test which tests that an error message is shown when a user tries to create a new user account by using an email address which is already associated with an existing user account, we should name out test method as follows: registerNewUserAccount_ExistingEmailAddressGiven_ShouldShowErrorMessage().

This naming convention ensures that:

  • The name of a test method describes a specific business or technical requirement.
  • The name of a test method describes expected input (or state) and the expected result for that input (state).

In other words, if we follow this naming convention we can answer to the following questions without reading the code of our test methods:

  • What are the features of our application?
  • What is the expected behavior of a feature or method when it receives an input X?

Also, if a test fails, we have a pretty good idea what is wrong before we read the source code of the failing test.

Pretty cool, huh?

Naming Test Class' Fields

A test class can have the following fields:

  • Fields which contains Test doubles such mocks or stubs.
  • A field which contains a reference to the tested object.
  • Fields which contains the other objects (testing utilities) which are used in our test cases.

We should name these fields by using the same rules which we use when we name the fields found from the application code. In other words, the name of each field should describe the "purpose" of the object which is stored to that field.

This rule sounds pretty "simple" (naming is always hard), and it has been easy for me to follow this rule when I name the tested class and the other classes which are used my tests. For example, if I have to add a TodoCrudService field to my test class, I use the name crudService.

When I have added fields which contain test doubles to my test class, I have typically added the type of the test double to the end of the field name. For example, if I have added a TodoCrudService mock to my test class, I have used the name crudServiceMock.

It sounds like a good idea but I have come to conclusion that it is a mistake. It is not a major problem but the thing is that a field name should describe the "purpose" of the field, not its type. Thus, we should not add the type of the test double to the field name.

Naming Local Variables

When we name the local variables used in our test methods, we should follow the same principles used when we name the variables found from our application code.

In my opinion, the most important rules are:

  • Describe the meaning of the variable. A good rule of thumb is that the variable name must describe the content of the variable.
  • Don’t use shortened names which aren’t obvious for anyone. Shortened names reduces readability and often you don’t gain anything by using them.
  • Don’t use generic names such as dto, modelObject, or data.
  • Be consistent. Follow the naming conventions of the used programming language. If your project has its own naming conventions, you should honor them as well.

Enough with theory. Let's put these lessons into practice.

Putting Theory into Practice

Let's take a look at a modified unit test (I made it worse) which is found from the example application of my Spring Social tutorial.

This unit test is written to test the registerNewUserAccount() method of the RepositoryUserService class, and it verifies that this method is working correctly when a new user account is created by using a social sign provider and a unique email address.

The source code of our test class looks as follows:

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.invocation.InvocationOnMock;
import org.mockito.runners.MockitoJUnitRunner;
import org.mockito.stubbing.Answer;
import org.springframework.security.crypto.password.PasswordEncoder;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import static org.mockito.Matchers.isA;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.mockito.Mockito.verifyZeroInteractions;
import static org.mockito.Mockito.when;

@RunWith(MockitoJUnitRunner.class)
public class RepositoryUserServiceTest {

    private RepositoryUserService service;

    @Mock
    private PasswordEncoder passwordEncoderMock;

    @Mock
    private UserRepository repositoryMock;

    @Before
    public void setUp() {
        service = new RepositoryUserService(passwordEncoderMock, repositoryMock);
    }


    @Test
    public void registerNewUserAccountByUsingSocialSignIn() throws DuplicateEmailException {
        RegistrationForm form = new RegistrationForm();
        form.setEmail("john.smith@gmail.com");
        form.setFirstName("John");
        form.setLastName("Smith");
        form.setSignInProvider(SocialMediaService.TWITTER);

        when(repositoryMock.findByEmail("john.smith@gmail.com")).thenReturn(null);
        
        when(repositoryMock.save(isA(User.class))).thenAnswer(new Answer<User>() {
            @Override
            public User answer(InvocationOnMock invocation) throws Throwable {
                Object[] arguments = invocation.getArguments();
                return (User) arguments[0];
            }
        });

        User modelObject = service.registerNewUserAccount(form);

        assertEquals("john.smith@gmail.com", modelObject.getEmail());
        assertEquals("John", modelObject.getFirstName());
        assertEquals("Smith", modelObject.getLastName());
        assertEquals(SocialMediaService.TWITTER, modelObject.getSignInProvider());
        assertEquals(Role.ROLE_USER, modelObject.getRole());
        assertNull(modelObject.getPassword());

        verify(repositoryMock, times(1)).findByEmail("john.smith@gmail.com");
        verify(repositoryMock, times(1)).save(modelObject);
        verifyNoMoreInteractions(repositoryMock);
        verifyZeroInteractions(passwordEncoderMock);
    }
}

This unit test has quite many problems:

  • The field names are pretty generic, and they describe the types of the test doubles.
  • The name of the test method is “pretty good” but it doesn’t describe the given input or the expected behavior.
  • The variable names used in the test method are awful.

We can improve the readability of this unit test by making the following changes to it:

  1. Change the name of the RepositoryUserService field to registrationService (the name of the service class is a bit bad but let’s ignore that).
  2. Remove the the word 'mock' from field names of the PasswordEncoder and UserRepository fields.
  3. Change the name of the test method to: registerNewUserAccount_SocialSignInAndUniqueEmail_ShouldCreateNewUserAccountAndSetSignInProvider().
  4. Change the name of the form variable to registration.
  5. Change the name of the modelObject variable to createdUserAccount.

The source code of our "modified" unit test looks as follows:

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.invocation.InvocationOnMock;
import org.mockito.runners.MockitoJUnitRunner;
import org.mockito.stubbing.Answer;
import org.springframework.security.crypto.password.PasswordEncoder;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import static org.mockito.Matchers.isA;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.mockito.Mockito.verifyZeroInteractions;
import static org.mockito.Mockito.when;


@RunWith(MockitoJUnitRunner.class)
public class RepositoryUserServiceTest {

    private RepositoryUserService registrationService;

    @Mock
    private PasswordEncoder passwordEncoder;

    @Mock
    private UserRepository repository;

    @Before
    public void setUp() {
        registrationService = new RepositoryUserService(passwordEncoder, repository);
    }


    @Test
    public void registerNewUserAccount_SocialSignInAndUniqueEmail_ShouldCreateNewUserAccountAndSetSignInProvider() throws DuplicateEmailException {
        RegistrationForm registration = new RegistrationForm();
        registration.setEmail("john.smith@gmail.com");
        registration.setFirstName("John");
        registration.setLastName("Smith");
        registration.setSignInProvider(SocialMediaService.TWITTER);

        when(repository.findByEmail("john.smith@gmail.com")).thenReturn(null);

        when(repository.save(isA(User.class))).thenAnswer(new Answer<User>() {
            @Override
            public User answer(InvocationOnMock invocation) throws Throwable {
                Object[] arguments = invocation.getArguments();
                return (User) arguments[0];
            }
        });

        User createdUserAccount = registrationService.registerNewUserAccount(registration);

        assertEquals("john.smith@gmail.com", createdUserAccount.getEmail());
        assertEquals("John", createdUserAccount.getFirstName());
        assertEquals("Smith", createdUserAccount.getLastName());
        assertEquals(SocialMediaService.TWITTER, createdUserAccount.getSignInProvider());
        assertEquals(Role.ROLE_USER, createdUserAccount.getRole());
        assertNull(createdUserAccount.getPassword());

        verify(repository, times(1)).findByEmail("john.smith@gmail.com");
        verify(repository, times(1)).save(createdUserAccount);
        verifyNoMoreInteractions(repository);
        verifyZeroInteractions(passwordEncoder);
    }
}

It is clear that this test case still has some problems but I think that our changes improved its readability. I think that the most dramatic improvements are:

  1. The name of test method describes the expected behavior of the tested method when a new user account is created by using a social sign in provider and a unique email address. The only way we could get this information from the “old” test case was to read the source code of the test method. This is obviously a lot slower than reading just the method name. In other words, giving good names to test methods saves time and helps us to get a quick overview about the requirements of the tested method or feature.
  2. the other changes transformed a generic CRUD test into a "use case". The "new" test method describes clearly
    1. What steps does this use case have.
    2. What the registerNewUserAccount() method returns when it receives a registration, which is made by using a social sign in provider and has a unique email address.

    In my opinion, the "old" test case failed to do this.

Summary

We have now learned that naming can have a huge positive effect to the readability of our test cases. We have also learned a few basic rules which helps us to transform our test cases into executable specifications.

However, our test case still has some problems. These problems are:

  • The test case uses magic numbers. We can make it better by replacing these magic numbers with constants.
  • The code which creates new RegistrationForm objects simply sets the property values of the created object. We can make this code better by using test data builders.
  • The standard JUnit assertions, which verify that the information of the returned User object is correct, are not very readable. Another problem is that they only check that the property values of the returned User object are correct. We can improve this code by turning assertions into a domain-specific language.

I will describe these techniques in the future.

In the meantime, I would love to hear what kind of naming conventions do you use.

15 comments… add one
  • Mitch May 19, 2014 @ 16:17

    Your naming conventions are good. Apart from the obvious benefit of knowing what the method does, it might also prevent "overloading" of methods with additional testing code. In our projects this really is a big issue, not so much in unit-tests but in normal code.

    I recently read about and used another strategy: (I thought it was linked on this site, but I don't remember...)
    The main test class contains static inner classes for each method. And the methods of those classes describe what is being testet.

    
    @RunWith(Enclosed.class)
    public class LinkUtilTest {
        public static class PrepareLinksForStorageMethod {
            public void changeLinkIfReplacementsAreDefined() {
                // test code
            }
        }
    }
    
    

    This gives nice grouping and testing output tells you really well what is going wrong.

    • Petri May 19, 2014 @ 21:02

      That looks interesting. I will try to find the blog post which describes this approach (although I cannot remember linking to it). If I find it, I will add a comment to this blog post.

      Anyway, there is another quite popular naming strategy which is described in a blog post titled Names Should Be Expressive When Writing Tests. If you are interested in this stuff, you should definitely read that blog post.

    • Lingviston Sep 4, 2016 @ 15:30

      I eventually came to the same idea. But it has some issues. For example, Android Studio doesn't catch up the outer class as a test case thus making it impossible to run all of the inner classes' tests.

      • Petri Sep 5, 2016 @ 21:21

        Hi,

        Have you tired using the junit-hierarchicalcontextrunner? I am using IntelliJ Idea and if I use this test runner, I can run a single unit test or all unit tests that belong to a specific inner class.

        However, I have to admit that I don't know if you can this if you are doing Android development.

  • Rafał May 24, 2014 @ 17:51

    Great post, as usual.

    In my opinion it is better and more readable to name the tests by features they are testing instead. One problem I see with the name like:

    registerNewUserAccount_SocialSignInAndUniqueEmail_ShouldCreateNewUserAccountAndSetSignInProvider

    is that it actually contains the method name of the class that it is testing. In case you refactor your service and change the method name, the test will need to be refactored too. To be honest, I started with this way of naming tests some time ago, than I switched to "should" and now I prefer just plain sentences.

    The other thing I don't like about this name is that it is not really a sentence. The name will also become longer as the method name is always included.

    On the other hand, I think this is matter of preference.

    The way described by @Mitch has also a lot of value (I heard about it couple of times, but never actually used it). I was thinking about mixing my approach with the static classes to have even better names (something to consider):

    
    class UserServiceTest {
       static class Registration {
         @Test
         registersNewUserWhenUniqueEmailAndPasswordsGiven () { }
         @Test
         throwsAnExceptionWhenPasswordsDoNotMatch() { }
       }
    }
    

    I blogged about naming tests by features some time ago: http://blog.codeleak.pl/2013/07/unit-test-names-describe-features.html

    • Petri May 25, 2014 @ 20:57

      Thank you for a good comment!

      I think that naming is an interesting topic for discussion because most of the time it is a matter of preference. Also, I have noticed that I learn from every discussion.

      I agree that this naming convention can lead into rather long method names and that renaming the tested method can be problematic (although Idea makes this a bit easier).

      But I think that names of unit test methods should identify the tested method because of two reasons:

      • It helps anyone to figure out where the problem is if a test fails (without reading the code of the test method).
      • Unit tests don't test a single feature. They test a method of a unit.

      On the other hand, if I write end-to-end tests, I identify the tested feature because in this case the name of the entry method is not important.

      Nevertheless, I have to admit that I am not entirely happy with this naming convention either. The biggest problem of it is that it isn't always easy to figure out a good description for expected input / current state. This was easy to do when I used the naming convention described in this blog post (start test names with should). The problem of this approach was it wasn't easy to describe the tested method or feature.

      Your comment made me think that maybe I should combine these two naming conventions.

      Now that I think about it, the technique suggested by @Mitch looks very interesting. I promise that I will try it and report my experiences later (it is a great topic for a new blog post).

  • Felipe Carvalho Apr 6, 2016 @ 13:23

    About "Naming Test Methods" section, I really like the idea of standardizing useful information to be present on tests names, but I personally don't like the convention you presented. As a reader, I find it bumpy to read, I usually am keen on test names that are closer to natural language.

    Also, after having seen this presentation Franziska Sauerwein, it made me rethink the way i name test classes and methods: http://slides.com/franziskasauerwein/outside#/13

    I really like the idea of using "Should" as a suffix for test classes. I find it more meaningful than plain old "Test", and, upon writing test methods, it helps bearing in mind that test names should describe the expectations we have about a given piece of code.

    This is a very nice series of posts, I'm having serious trouble to stop reading it!

  • Fernando Jul 29, 2016 @ 16:25

    This is a great post! Petri I am following most of your stuff and It's working great in my projects.
    Nowadays I am looking examples of how to deal with huge tests (i mean when you have to mock many things, and there are many outputs possibles, maybe because of business validation. I suppose that test are huge because of a bad implementation of the code. But it's difficult to me to think in another way.

    Sorry for my english! Thank you for your time!

    • Petri Aug 2, 2016 @ 13:39

      Hi Fernando!

      First, your English is fine. Don't worry about it.

      Second, if you need to write huge tests, I recommend that you take a look at this blog post. It explains how you can write clean tests by using nested inner classes. However, nowadays I use a different library for that purpose. I will release a new version of that post as a free demo of my Test With Spring course. So, you might want to check it out when it is done.

  • Ben Feb 25, 2019 @ 17:26

    May I ask why the first letter in your test methods is always lower case? Is there a reason for that? Why not make it upper case? Like this:

    registerNewUserAccount_ExistingEmailAddressGiven_ShouldThrowException().

    Why not make it:

    RegisterNewUserAccount_ExistingEmailAddressGiven_ShouldThrowException().

    • Petri Feb 25, 2019 @ 17:36

      Hi,

      The reason is that the "original" Java coding convention has a naming convention for method names which states that the first letter of a method name should use lower case. In other words, I am just used to this naming convention (this could be problem if I would use .NET because it seems to use a different naming convention).

      • Ben Feb 26, 2019 @ 18:08

        Thanks!

Leave a Reply