I released the starter package of my Test With Spring course. Take a look at the course >>

Spring Batch Tutorial: Creating a Custom ItemReader

pink data concept

Spring Batch has a good support for reading input data from different data sources such as files and databases.

However, it is quite common that we have to read input data from a data source that is not supported out of the box. This means that we have to implement a component that reads the input data from our data source.

This blog post helps us to solve that problem and describes how we can create a custom ItemReader.

Let’s get started.

If you are not familiar with Spring Batch, you should read the following blog posts before you continue reading this blog post:

Creating a Custom ItemReader

We can create a custom ItemReader by following these steps:

  1. Create a class that implements the ItemReader<T> interface and provide the type of the returned object as a type parameter.
  2. Implement the T read() method of the ItemReader<T> interface by following these rules:
    • The read() method returns an object that contains the information of the next item.
    • If the next item is not found, the read() method must return null.

Let’s create a custom ItemReader that returns the student information of an online testing course as StudentDTO objects that are read from the memory.

The StudentDTO class is a simple data transfer object, and its source code looks as follows:

public class StudentDTO {
 
    private String emailAddress;
    private String name;
    private String purchasedPackage;
 
    public StudentDTO() {}
 
    public String getEmailAddress() {
        return emailAddress;
    }
 
    public String getName() {
        return name;
    }
 
    public String getPurchasedPackage() {
        return purchasedPackage;
    }
 
    public void setEmailAddress(String emailAddress) {
        this.emailAddress = emailAddress;
    }
 
    public void setName(String name) {
        this.name = name;
    }
 
    public void setPurchasedPackage(String purchasedPackage) {
        this.purchasedPackage = purchasedPackage;
    }
}

We can create our ItemReader by following these steps:

  1. Create an InMemoryStudentReader class.
  2. Implement the ItemReader interface and set the type of the returned object to StudentDTO
  3. Add a List<Student> field into the created class. This field contains the student information of the
    course.
  4. Add an int field called nextStudentIndex into the class. This field contains the index of the next StudentDTO object.
  5. Add a private initialize() method to created class. This method creates the student data and sets the index of the next student to 0.
  6. Create a constructor that invokes the initialize() method.
  7. Implement the read() method by following these rules:
    • If the next student is found, return the found StudentDTO object and increase the index of the next student by 1.
    • If the next student is not found, return null.

The source code of the InMemoryStudentReader class looks as follows:

import org.springframework.batch.item.ItemReader;

import java.util.Arrays;
import java.util.Collections;
import java.util.List;

public class InMemoryStudentReader implements ItemReader<StudentDTO> {

    private int nextStudentIndex;
    private List<StudentDTO> studentData;

    InMemoryStudentReader() {
        initialize();
    }

    private void initialize() {
        StudentDTO tony = new StudentDTO();
        tony.setEmailAddress("tony.tester@gmail.com");
        tony.setName("Tony Tester");
        tony.setPurchasedPackage("master");

        StudentDTO nick = new StudentDTO();
        nick.setEmailAddress("nick.newbie@gmail.com");
        nick.setName("Nick Newbie");
        nick.setPurchasedPackage("starter");

        StudentDTO ian = new StudentDTO();
        ian.setEmailAddress("ian.intermediate@gmail.com");
        ian.setName("Ian Intermediate");
        ian.setPurchasedPackage("intermediate");

        studentData = Collections.unmodifiableList(Arrays.asList(tony, nick, ian));
        nextStudentIndex = 0;
    }

    @Override
    public StudentDTO read() throws Exception {
        StudentDTO nextStudent = null;

        if (nextStudentIndex < studentData.size()) {
            nextStudent = studentData.get(nextStudentIndex);
            nextStudentIndex++;
        }

        return nextStudent;
    }
}

After we have created our custom ItemReader class, we have to configure the ItemReader bean that provides the input data for our Spring Batch job. Let’s find out how we can do it.

Configuring the ItemReader Bean

We can configure our ItemReader bean by following these steps:

  1. Create a configuration class and annotate the created class with the @Configuration annotation.
  2. Add a new method to the created class, ensure that this method returns an ItemReader<StudentDTO> object, and annotate this method with the @Bean annotation.
  3. Implement the method by returning a new InMemoryStudentReader object.

The source code of our configuration class looks as follows:

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class InMemoryStudentJobConfig {

    @Bean
    ItemReader<StudentDTO> inMemoryStudentReader() {
        return new InMemoryStudentReader();
    }
}

Let’s summarize what we learned from this blog post.

Summary

This blog post has taught us three things:

  • We can create a custom ItemReader by implementing the ItemReader<T> interface.
  • When we implement the ItemReader<T> interface, we have to provide the type of the returned object as a type parameter.
  • The T read() method of the ItemReader<T> interface must return the next T object. If the next object is not found, it must return null.

The next part of this tutorial describes how we create a custom ItemReader that reads the input data of our batch job by using an external REST API.

P.S. You can get the example applications of this blog post from Github: Spring example and Spring Boot example.

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He is specialized in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

17 comments… add one
  • thank you

    Reply
    • You are welcome!

      Reply
  • Hi Petri,

    I have come across a scenario, where I have declared one exception in skippable exception, to skip the invalid records detected in item processor. Could you please help me with one possible way through which I can keep track of the all skipped records and at the end using job execution listener send a mail of all those skipped records.

    Any suggestions are appreciated.

    Reply
    • Hi Naveen,

      You can use a SkipListener for this purpose. It seems that you should implement your own SkipListener that keeps track of the skipped records (for example, it can save them to database). When the job is finished, you can simply fetch the skipped records and send the email that contains the required information.

      Reply
    • Hi Petri,
      I am using SpringBoot. my integration context xml file has int:jdbc inbound channel where I give database select query. I want Spring batch to load multiple table/multi records and process and write to XML output. Please give me some suggetion

      Reply
      • Hi,

        Take a look at this blog post. It explains how you can use the Spring Integration JDBC inbound channel adapter.

        Reply
  • Hi there,

    I have tried this but am having problems when I run my code with an IllegalArgumentException – Assertion Failure – this argument is required; it must not be null.

    Any ideas what could be causing that?

    Reply
    • Hi,

      Did you try to create a batch job that doesn’t have a reader and a writer?

      Reply
  • I have batch flow which has a reader and a writer. The reader is jdbccursorItemReader. The writer is custom Itemwriter which has a flatfileitemwriter property.

    My question is, if the reader, does not fetch any row, will the writer still execute?

    PS: in case the reader fetches 0 records i have to write an empty file using flatfileitemwriter.

    Reply
    • Hi,

      If the reader cannot find the next input record, its read() must return null. This means that the writer won’t write any lines to the file if the reader cannot find any records from the database. That being said, it does create the file and doesn’t delete it even if it is empty (as long as you use the default settings).

      Reply
  • @Petri,
    What about the thread safety of primitives used in ItemReader(nextStudentIndex )

    Reply
    • Well, since you asked that question, I assume that you realized that the example is not thread safe. If you need to ensure that the primitives are thread safe, you need to replace the int variable (nextStudentIndex) with AtomicInteger.

      Reply

Leave a Comment