Introduction to Spies

When we are writing automated tests for our code, often we notice that it's not possible to invoke the real dependencies of the system under test. The problem might be that:

• The problematic dependency invokes an external API which cannot be accessed from our test environment.
• We cannot invoke the real dependency because invoking it would cause unwanted side effects.
• The real dependency is too slow and invoking it would slow down our test suite.

If we cannot use the real dependency, we have to replace it with a test double that must provide the same API as the replaced dependency. This ensures that the system under test will think that it's interacting with the real thing.

There are multiple different test doubles and each test double helps us to solve a very specific problem. This time we will take a closer look at a test double called a spy.

After we have finished this blog post, we:

• Know what a spy is.
• Can create spies from the scratch and use them in our automated tests.
• Understand when we should use spies.

Let's begin.

What Is a Spy

A spy is a test double that can replace an external dependency of the system under test. A spy must fulfill these requirements:

• A spy must provide the same API as the replaced dependency. This means that if the external dependency is a class, our spy must extend it and override all methods. On the other hand, if the replaced dependency is an interface, our spy must implement the replaced interface.
• A spy provides no way to configure the response that's returned when its methods are invoked.
• A spy doesn't allow us to specify expectations for the arguments which are passed to the invoked methods.
• A spy might record the method invocations of its methods and the arguments which were passed to the invoked methods, but it doesn’t have to record both of these things.
• A spy doesn't provide a direct way to verify the interactions that happened between the system under test and the spy. Instead, we have to use "getter" methods to access the information that was recorded by the spy and write assertions for the information returned by these methods.

Next, we will put theory into practice and create a new spy. Let's start by taking a quick look at the system under test.

Introduction to the System Under Test

The system under test has two dependencies:

First, the TodoItemRepository interface declares one method (create()) which is used to insert new todo items into the database. The create() method takes the information of the created todo item as a method parameter and returns the information that was inserted into the database.

The source code of the TodoItemRepository interface looks as follows:

interface TodoItemRepository {

    TodoItem create(CreateTodoItem input);
}

The CreateTodoItem class contains the information of the created todo item. Its source code looks as follows:

public class CreateTodoItem {

    private String title;

    //Getters and setters are omitted
}

The TodoItem class contains the information of a single todo item. Its source code looks as follows:

class TodoItem {

    private Long id;
    private String title;

    //Getters and setters are omitted
}

Second, the TodoItemAuditLog interface declares one method (logTodoItem()) which keeps track of changes made to todo items by writing new rows to an audit log. The logTodoItem() method doesn't return anything and it takes three method parameters:

1. The id of the user who performed the action.
2. The performed action.
3. The target object (a TodoItem object).

The source code of the TodoItemAuditLog interface looks as follows:

interface TodoItemAuditLog {

    void logTodoItem(Long userId, AuditLogAction action, TodoItem todoItem);
}

The AuditLogAction enum specifies the actions which can be performed to a todo item. Its source code looks as follows:

enum AuditLogAction {

    CREATE,
    DELETE,
    UPDATE;
}

Let's assume that we have to write unit tests for the create() method of the TodoItemService class. The implementation of this method follows these steps:

1. Insert a new todo item into the database.
2. Write an entry to the audit log.
3. Return the information that was inserted into the database.

The source code of the TodoItemService class class looks as follows:

public class TodoItemService {

    private final TodoItemAuditLog auditLog;
    private final TodoItemRepository repository;

    public TodoItemService(TodoItemAuditLog auditLog, TodoItemRepository repository) {
        this.auditLog = auditLog;
        this.repository = repository;
    }

    public TodoItem create(Long userId, CreateTodoItem input) {
        var created = repository.create(input);
        auditLog.logTodoItem(userId, AuditLogAction.CREATE, created);
        return created;
    }
}

Let's move on and find out how we can create a simple TodoItemAuditLog spy.

Creating a Simple Spy

When we want to create a spy that can replace the TodoItemAuditLog dependency, we have to follow these steps:

First, we have to create a class that contains all arguments which are passed to the logTodoItem() method of the TodoItemAuditLog interface when it's invoked by the system under test. After we have created this class, its source code looks as follows:

class MethodInvocationArguments {

    private final AuditLogAction action;
    private final TodoItem todoItem;
    private final Long userId;

    MethodInvocationArguments(Long userId, AuditLogAction action, TodoItem todoItem) {
        this.action = action;
        this.todoItem = todoItem;
        this.userId = userId;
    }

    //Getters are omitted
}

Second, we have to create a new class and ensure that this class implements the TodoItemAuditLog interface. After we have created our spy class, its source code looks as follows:

class TodoItemAuditLogSpy implements TodoItemAuditLog {

    @Override
    public void logTodoItem(Long userId, AuditLogAction action, TodoItem todoItem) {
        //Implementation left blank on purpose
    }
}

Third, we have to add one field to the TodoItemAuditLogSpy class. The final arguments field contains the arguments which are passed to the logTodoItem() method. After we have added this field to the TodoItemAuditLogSpy class, its source code looks as follows:

import java.util.ArrayList;
import java.util.List;

class TodoItemAuditLogSpy implements TodoItemAuditLog {

    private final List<MethodInvocationArguments> arguments = new ArrayList<>();

    @Override
    public void logTodoItem(Long userId, AuditLogAction action, TodoItem todoItem) {
        //Implementation left blank on purpose
    }
}

Fourth, we have to implement the logTodoItem() method. Our implementation simply creates a new MethodInvocationArguments object that contains the arguments passed to the logTodoItem() method and adds the created object to the arguments list.

After we have implemented the logTodoItem() method, the source code of our spy class looks as follows:

import java.util.ArrayList;
import java.util.List;

class TodoItemAuditLogSpy implements TodoItemAuditLog {

    private final List<MethodInvocationArguments> arguments = new ArrayList<>();

    @Override
    public void logTodoItem(Long userId, AuditLogAction action, TodoItem todoItem) {
        var invocationArguments = new MethodInvocationArguments(userId, action, todoItem);
        arguments.add(invocationArguments);
    }
}

Fifth, we have to write the methods which allow us to access the information recorded by our spy. These methods are:

• The getMethodInvocationCount() method returns the number of recorded method invocations.
• The getArgumentsForMethodInvocation() method takes the index (zero-based) of the method invocation as a method parameter and returns a MethodInvocationArguments object that contains all arguments of the requested method invocation. If the requested arguments aren't found (the index is too big), this method throws a NoArgumentsFoundException.

After we have written these methods, the source code of our spy class looks as follows:

import java.util.ArrayList;
import java.util.List;

class TodoItemAuditLogSpy implements TodoItemAuditLog {

    private final List<MethodInvocationArguments> arguments = new ArrayList<>();

    @Override
    public void logTodoItem(Long userId, AuditLogAction action, TodoItem todoItem) {
        var invocationArguments = new MethodInvocationArguments(userId, action, todoItem);
        arguments.add(invocationArguments);
    }

    int getMethodInvocationCount() {
        return arguments.size();
    }

    MethodInvocationArguments getArgumentsForMethodInvocation(int methodInvocationIndex) {
        if (methodInvocationIndex > arguments.size() - 1) {
            throw new NoArgumentsFoundException(
                    "No arguments found with index: %d. Found %d method invocations",
                    methodInvocationIndex,
                    arguments.size()
            );
        }
        return arguments.get(methodInvocationIndex);
    }
}

The source code of the NoArgumentsFoundException class looks as follows:

class NoArgumentsFoundException extends RuntimeException {

    NoArgumentsFoundException(String messageTemplate, Object... arguments) {
        super(String.format(messageTemplate, arguments));
    }
}

We have now written a simple spy. Let's move on and write a few test methods which use our new spy.

Using Our New Spy

We can use our new spy by following these steps:

First, we have to create a new spy and replace the TodoItemAuditLog dependency of the system under test with the created spy. Because spies aren't stateless, we have to create a new spy before a test method is invoked. In other words, we have to add a new setup method to our test class and annotate this method with the @BeforeEach annotation. After we have added a new setup method to our test class, we have to implement it by following these steps:

1. Create a new CreateTodoItem object which contains the information of the created todo item.
2. Create a new TodoItemAuditLogSpy object.
3. Create a new TodoItemRepositoryFake object. This fake puts the created todo items to a map.
4. Create a new TodoItemService object and pass the required dependencies as constructor arguments.

After we have written our setup method, the source code of our test class looks as follows:

class TodoItemServiceTest {

    private final String TITLE = "Write a new blog post";

    private TodoItemAuditLogSpy auditLog;
    private TodoItemRepository repository;
    private TodoItemService service;

    private CreateTodoItem input;

    @BeforeEach
    void configureSystemUnderTest() {
        createInput();
        auditLog = new TodoItemAuditLogSpy();
        repository = new TodoItemRepositoryFake();
        service = new TodoItemService(auditLog, repository);
    }

    private void createInput() {
        input = new CreateTodoItem();
        input.setTitle(TITLE);
    }
}

Second, we can now write the test methods which use our spy by following these steps:

1. Ensure that the system under test writes one entry to the audit log.
2. Verify that the new audit log entry contains the correct user id.
3. Ensure that the new audit log entry contains the correct action.
4. Verify that the new audit log entry contains the correct todo item.
5. Ensure that the system under test returns the information of the created todo item.

After we have written the required test methods, the source code of our test class looks as follows:

mport org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;

import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.SoftAssertions.assertSoftly;

class TodoItemService2Test {

    private final Long EXPECTED_ID = 1L;
    private final String TITLE = "Write a new blog post";
    private final Long USER_ID = 5L;

    private TodoItemAuditLogSpy auditLog;
    private TodoItemRepository repository;
    private TodoItemService service;

    private CreateTodoItem input;

    @BeforeEach
    void configureSystemUnderTest() {
        createInput();
        auditLog = new TodoItemAuditLogSpy();
        repository = new TodoItemRepositoryFake();
        service = new TodoItemService(auditLog, repository);
    }

    private void createInput() {
        input = new CreateTodoItem();
        input.setTitle(TITLE);
    }

    @Test
    @DisplayName("Should write one entry to the audit log")
    void shouldWriteOneEntryToAuditLog() {
        service.create(USER_ID, input);

        var methodInvocationCount = auditLog.getMethodInvocationCount();
        assertThat(methodInvocationCount).isEqualByComparingTo(1);
    }

    @Test
    @DisplayName("Should write a new audit log entry with the correct user id")
    void shouldWriteNewAuditLogEntryWithCorrectUserId() {
        service.create(USER_ID, input);

        var methodInvocationArguments = auditLog.getArgumentsForMethodInvocation(0);
        assertThat(methodInvocationArguments.getUserId())
                .isEqualByComparingTo(USER_ID);
    }

    @Test
    @DisplayName("Should write a new audit log entry with the correct action")
    void shouldWriteNewAuditLogEntryWithCorrectAction() {
        service.create(USER_ID, input);

        var methodInvocationArguments = auditLog.getArgumentsForMethodInvocation(0);
        assertThat(methodInvocationArguments.getAction())
                .isEqualByComparingTo(AuditLogAction.CREATE);
    }

    @Test
    @DisplayName("Should write a new audit log entry with the correct todo item")
    void shouldWriteNewAuditLogEntryWithCorrectTodoItem() {
        service.create(USER_ID, input);

        var methodInvocationArguments = auditLog.getArgumentsForMethodInvocation(0);
        assertSoftly((softAssertions -> {
            var todoItem = methodInvocationArguments.getTodoItem();

            softAssertions.assertThat(todoItem.getId())
                    .as("id")
                    .isEqualByComparingTo(EXPECTED_ID);
            softAssertions.assertThat(todoItem.getTitle())
                    .as("title")
                    .isEqualTo(TITLE);
        }));
    }

    @Test
    @DisplayName("Should return a todo item with the correct information")
    void shouldReturnTodoItemWithCorrectInformation() {
        var returned = service.create(USER_ID, input);

        assertSoftly((softAssertions -> {
            softAssertions.assertThat(returned.getId())
                    .as("id")
                    .isEqualByComparingTo(EXPECTED_ID);
            softAssertions.assertThat(returned.getTitle())
                    .as("title")
                    .isEqualTo(TITLE);
        }));
    }
}

Next, we will identify the situations when we should use spies.

When Should We Use Spies?

We should use a spy if any of the following conditions is true:

• We cannot specify our expectations at compile-time. If we cannot predict the expected interactions before the system under test is run, it's often cleaner to use a spy and write assertions for the values recorded by our spy.
• A failed expectation cannot be reported back to the test runner. This might happen if the system under test is running in a different thread or process than the test which invokes it.
• We want to specify the error message that's shown if our expectations aren't met. Typically mocking frameworks won't allow us to customize the error message that's shown if our expectations aren't met. Thus, if we want to use a custom error message, we have to use a spy and write assertions for the values recorded by our spy.
• We want to use assertions because they reveal our intentions better than using a mock object. Even though modern mocking frameworks (such as Mockito or Mockk) have flexible APIs, these APIs aren't good at emphasizing business rules. Thus, if we want to emphasize business rules when we specify our expectations, we should use a spy instead of a mock and write assertions for the values recorded by our spy.

At this point, we should understand how a spy works and know when we should replace a dependency of the system under test with a spy. Let's summarize what we learned from this blog post.

Summary

This blog post has taught us six things:

• A spy must provide the same API as the replaced dependency.
• A spy provides no way to configure the response that's returned when its methods are invoked.
• A spy doesn't allow us to specify expectations for the arguments which are passed to the invoked methods.
• A spy might record the method invocations of its methods and the arguments which were passed to the invoked methods.
• A spy doesn't provide a direct way to verify the interactions that happened between the system under test and the spy.
• We should replace a dependency with a spy if we cannot use a mock or if using a spy makes our test code cleaner than using a mock.

P.S. You can get the example application of this blog post from Github.