Writing Clean Tests - Replace Assertions with a Domain-Specific Language

Automated tests are worthless if they don’t assert anything, but the problem of regular JUnit assertions is that they speak the wrong language and become messy if we have to write a lot of them.

If we want to write tests which are both easy understand and maintain, we have to figure out a better way of writing assertions.

This blog post identifies the problems of the "standard" JUnit assertions, and describes how we solve these problems by replacing these assertions with a domain-specific language.

Data Is Not That Important

In my previous blog post I identified two problems caused by data centric tests. Although that blog post talked about the creation of new objects, these problems are valid for assertions as well.

Let’s refresh our memory and take a look at the source code of our unit test which ensures that the registerNewUserAccount(RegistrationForm userAccountData) method of the RepositoryUserService class works as expected when a new user account is created by using a unique email address and a social sign in provider.

Our unit test looks as follows (the relevant code is highlighted):

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.invocation.InvocationOnMock;
import org.mockito.runners.MockitoJUnitRunner;
import org.mockito.stubbing.Answer;
import org.springframework.security.crypto.password.PasswordEncoder;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import static org.mockito.Matchers.isA;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.mockito.Mockito.verifyZeroInteractions;
import static org.mockito.Mockito.when;

@RunWith(MockitoJUnitRunner.class)
public class RepositoryUserServiceTest {

	private static final String REGISTRATION_EMAIL_ADDRESS = "john.smith@gmail.com";
	private static final String REGISTRATION_FIRST_NAME = "John";
	private static final String REGISTRATION_LAST_NAME = "Smith";
	private static final Role ROLE_REGISTERED_USER = Role.ROLE_USER;
	private static final SocialMediaService SOCIAL_SIGN_IN_PROVIDER = SocialMediaService.TWITTER;

	private RepositoryUserService registrationService;

	@Mock
	private PasswordEncoder passwordEncoder;

	@Mock
	private UserRepository repository;

	@Before
	public void setUp() {
		registrationService = new RepositoryUserService(passwordEncoder, repository);
	}


	@Test
	public void registerNewUserAccount_SocialSignInAndUniqueEmail_ShouldCreateNewUserAccountAndSetSignInProvider() throws DuplicateEmailException {
		RegistrationForm registration = new RegistrationFormBuilder()
			.email(REGISTRATION_EMAIL_ADDRESS)
			.firstName(REGISTRATION_FIRST_NAME)
			.lastName(REGISTRATION_LAST_NAME)
			.isSocialSignInViaSignInProvider(SOCIAL_SIGN_IN_PROVIDER)
			.build();

		when(repository.findByEmail(REGISTRATION_EMAIL_ADDRESS)).thenReturn(null);

		when(repository.save(isA(User.class))).thenAnswer(new Answer<User>() {
			@Override
			public User answer(InvocationOnMock invocation) throws Throwable {
				Object[] arguments = invocation.getArguments();
				return (User) arguments[0];
			}
		});

		User createdUserAccount = registrationService.registerNewUserAccount(registration);

		assertEquals(REGISTRATION_EMAIL_ADDRESS, createdUserAccount.getEmail());
		assertEquals(REGISTRATION_FIRST_NAME, createdUserAccount.getFirstName());
		assertEquals(REGISTRATION_LAST_NAME, createdUserAccount.getLastName());
		assertEquals(SOCIAL_SIGN_IN_PROVIDER, createdUserAccount.getSignInProvider());
		assertEquals(ROLE_REGISTERED_USER, createdUserAccount.getRole());
		assertNull(createdUserAccount.getPassword());

		verify(repository, times(1)).findByEmail(REGISTRATION_EMAIL_ADDRESS);
		verify(repository, times(1)).save(createdUserAccount);
		verifyNoMoreInteractions(repository);
		verifyZeroInteractions(passwordEncoder);
	}
}

As we can see, the assertions found from our unit test ensures that the property values of the returned User object are correct. Our assertions ensure that:

  • The value of the email property is correct.
  • The value of the firstName property is correct.
  • The value of the lastName property is correct.
  • The value of the signInProvider is correct.
  • The value of the role property is correct.
  • The password is null.

This is of course pretty obvious but it is important to repeat these assertions in this way because it helps us to identify the problem of our assertions. Our assertions are data centric and this means that:

  • The reader has to know the different states of the returned object. For example, if we think about our example, the reader has to know that if the email, firstName, lastName, and signInProvider properties of returned RegistrationForm object have non-null values and the value of the password property is null, it means that the object is a registration which is made by using a social sign in provider.
  • If the created object has many properties, our assertions litters the source code of our tests. We should remember that even though we want to ensure that the data of the returned object is correct, it is much more important that we describe the state of the returned object.

Let’s see how we can improve our assertions.

Turning Assertions into a Domain-Specific Language

You might have noticed that often the developers and the domain experts use different terms for the same things. In other words, developers don’t speak the same language than the domain experts. This causes unnecessary confusion and friction between the developers and the domain experts.

Domain-driven design (DDD) provides one solution to this problem. Eric Evans introduced the term ubiquitous language in his book titled Domain-Driven Design.

Wikipedia specifies ubiquitous language as follows:

Ubiquitous language is a language structured around the domain model and used by all team members to connect all the activities of the team with the software.

If we want write assertions which speak the "correct" language, we have to bridge the gap between the developers and the domain experts. In other words, we have to create a domain-specific language for writing assertions.

Implementing Our Domain-Specific Language

Before we can implement our domain-specific language, we have to design it. When we design a domain-specific language for our assertions, we have to follow these rules:

  1. We have to abandon the data centric approach and think more about the real user whose information is found from a User object.
  2. We have to use the language spoken by the domain experts.
I won’t do into the details here because this is a huge topic and it is impossible to explain it in a single blog. If you want learn more about domain-specific languages and Java, you can get started by reading the following blog posts:

If we follow these two rules, we can create the following rules for our domain-specific language:

  • A user has a first name, last name, and email address.
  • A user is a registered user.
  • A user is registered by using a social sign provider which means that this user doesn’t have a password.

Now that we have specified the rules of our domain-specific language, we are ready to implement it. We are going to do this by creating a custom AssertJ assertion which implements the rules of our domain-specific language.

I will not describe the required steps in this blog post because I have written a blog post which describes them. If you are not familiar with AssertJ, I recommend that you read that blog post before reading the rest of this blog post.

The source code of our custom assertion class looks as follows:

import org.assertj.core.api.AbstractAssert;
import org.assertj.core.api.Assertions;

public class UserAssert extends AbstractAssert<UserAssert, User> {

    private UserAssert(User actual) {
        super(actual, UserAssert.class);
    }

    public static UserAssert assertThatUser(User actual) {
        return new UserAssert(actual);
    }

    public UserAssert hasEmail(String email) {
        isNotNull();

        Assertions.assertThat(actual.getEmail())
                .overridingErrorMessage( "Expected email to be <%s> but was <%s>",
                        email,
                        actual.getEmail()
                )
                .isEqualTo(email);

        return this;
    }

    public UserAssert hasFirstName(String firstName) {
        isNotNull();

        Assertions.assertThat(actual.getFirstName())
                .overridingErrorMessage("Expected first name to be <%s> but was <%s>",
                        firstName,
                        actual.getFirstName()
                )
                .isEqualTo(firstName);

        return this;
    }

    public UserAssert hasLastName(String lastName) {
        isNotNull();

        Assertions.assertThat(actual.getLastName())
                .overridingErrorMessage( "Expected last name to be <%s> but was <%s>",
                        lastName,
                        actual.getLastName()
                )
                .isEqualTo(lastName);

        return this;
    }

    public UserAssert isRegisteredByUsingSignInProvider(SocialMediaService signInProvider) {
        isNotNull();

        Assertions.assertThat(actual.getSignInProvider())
                .overridingErrorMessage( "Expected signInProvider to be <%s> but was <%s>",
                        signInProvider,
                        actual.getSignInProvider()
                )
                .isEqualTo(signInProvider);

        hasNoPassword();

        return this;
    }

    private void hasNoPassword() {
        isNotNull();

        Assertions.assertThat(actual.getPassword())
                .overridingErrorMessage("Expected password to be <null> but was <%s>",
                        actual.getPassword()
                )
                .isNull();
    }

    public UserAssert isRegisteredUser() {
        isNotNull();

        Assertions.assertThat(actual.getRole())
                .overridingErrorMessage( "Expected role to be <ROLE_USER> but was <%s>",
                        actual.getRole()
                )
                .isEqualTo(Role.ROLE_USER);

        return this;
    }
}

We have now created a domain-specific language for writing assertions to User objects. Our next step is to modify our unit test to use our new domain-specific language.

Replacing JUnit Assertions with a Domain-Specific Language

After we have rewritten our assertions to use our domain-specific language, the source code of our unit test looks as follows (the relevant part is highlighted):

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.mockito.Mock;
import org.mockito.invocation.InvocationOnMock;
import org.mockito.runners.MockitoJUnitRunner;
import org.mockito.stubbing.Answer;
import org.springframework.security.crypto.password.PasswordEncoder;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
import static org.mockito.Matchers.isA;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoMoreInteractions;
import static org.mockito.Mockito.verifyZeroInteractions;
import static org.mockito.Mockito.when;

@RunWith(MockitoJUnitRunner.class)
public class RepositoryUserServiceTest {

	private static final String REGISTRATION_EMAIL_ADDRESS = "john.smith@gmail.com";
	private static final String REGISTRATION_FIRST_NAME = "John";
	private static final String REGISTRATION_LAST_NAME = "Smith";
	private static final Role ROLE_REGISTERED_USER = Role.ROLE_USER;
	private static final SocialMediaService SOCIAL_SIGN_IN_PROVIDER = SocialMediaService.TWITTER;

	private RepositoryUserService registrationService;

	@Mock
	private PasswordEncoder passwordEncoder;

	@Mock
	private UserRepository repository;

	@Before
	public void setUp() {
		registrationService = new RepositoryUserService(passwordEncoder, repository);
	}


	@Test
	public void registerNewUserAccount_SocialSignInAndUniqueEmail_ShouldCreateNewUserAccountAndSetSignInProvider() throws DuplicateEmailException {
		RegistrationForm registration = new RegistrationFormBuilder()
			.email(REGISTRATION_EMAIL_ADDRESS)
			.firstName(REGISTRATION_FIRST_NAME)
			.lastName(REGISTRATION_LAST_NAME)
			.isSocialSignInViaSignInProvider(SOCIAL_SIGN_IN_PROVIDER)
			.build();

		when(repository.findByEmail(REGISTRATION_EMAIL_ADDRESS)).thenReturn(null);

		when(repository.save(isA(User.class))).thenAnswer(new Answer<User>() {
			@Override
			public User answer(InvocationOnMock invocation) throws Throwable {
				Object[] arguments = invocation.getArguments();
				return (User) arguments[0];
			}
		});

		User createdUserAccount = registrationService.registerNewUserAccount(registration);

		assertThatUser(createdUserAccount)
			.hasEmail(REGISTRATION_EMAIL_ADDRESS)
			.hasFirstName(REGISTRATION_FIRST_NAME)
			.hasLastName(REGISTRATION_LAST_NAME)
			.isRegisteredUser()
			.isRegisteredByUsingSignInProvider(SOCIAL_SIGN_IN_PROVIDER);

		verify(repository, times(1)).findByEmail(REGISTRATION_EMAIL_ADDRESS);
		verify(repository, times(1)).save(createdUserAccount);
		verifyNoMoreInteractions(repository);
		verifyZeroInteractions(passwordEncoder);
	}
}

Our solution has the following the benefits:

  • Our assertions use the language which is understood by the domain experts. This means that our test is an executable specification which is easy to understand and always up-to-date.
  • We don’t have to waste time for figuring out why a test failed. Our custom error messages ensure that we know why it failed.
  • If the API of the User class changes, we don’t have to fix every test method that writes assertions to User objects. The only class which we have to change is the UserAssert class. In other words, moving the actual assertions logic away from our test method made our test less brittle and easier to maintain.

Let’s spend a moment to summarize what we learned from this blog post.

Summary

We have now transformed our assertions into a domain-specific language. This blog post taught us three things:

  • Following the data centric approach causes unnecessary confusion and friction between the developers and the domain experts.
  • Creating a domain-specific language for our assertions makes our tests less brittle because the actual assertion logic is moved to custom assertion classes.
  • If we write assertions by using a domain-specific language, we transform our tests into executable specifications which are easy to understand and speak the language of the domain experts.
7 comments… add one
  • Stephane Jun 14, 2014 @ 17:01

    Thanks Petri ! That's a very interesting post of yours !

    • Petri Jun 15, 2014 @ 10:41

      Thank you for your kind words. I really appreciate them.

  • Vincent Fleetwood Aug 17, 2014 @ 0:54

    Very nice, as always. The tests read well and make sense.

    I found that the more complicated the scenarios become, the greater the benefit from a DSL.

    • Petri Aug 20, 2014 @ 20:10

      Thank you for your kind words. I really appreciate them.

      I found that the more complicated the scenarios become, the greater the benefit from a DSL.

      I agree. Sometimes the required assertions are so complicated that it is really hard to figure out what the expected outcome is. DSLs help us to fix this because we can move the assertion logic to the methods of our custom assertion class and name these methods so that the expected outcome is clear to domain experts (and of course to developers).

Leave a Reply