Spring Batch Tutorial: Writing Information to a CSV File

The previous parts of this tutorial have taught you to read the input data of your batch job from different data sources. However, your batch job isn't very useful because you don’t know how you can save the output data of your batch job.

This time you will learn to write the output data of your Spring Batch job to a CSV file. After you have read this blog post, you:

  • Know how you can add a header line to a CSV file.
  • Understand how you can transform your domain object into a line that's written to a CSV file.
  • Can write the output data of your batch job to a CSV file.

Let’s start by taking a quick look at your batch job.

This blog post assumes that:

Introduction to Your Batch Job

The example batch job of this blog post processes the student information of an online course. The input data of this batch job is read from a data source and transformed into StudentDTO objects. The StudentDTO class contains the information of a single student, and its source code looks as follows:

public class StudentDTO {
  
    private String emailAddress;
    private String name;
    private String purchasedPackage;
  
    public StudentDTO() {}
  
    public String getEmailAddress() {
        return emailAddress;
    }
  
    public String getName() {
        return name;
    }
  
    public String getPurchasedPackage() {
        return purchasedPackage;
    }
  
    public void setEmailAddress(String emailAddress) {
        this.emailAddress = emailAddress;
    }
  
    public void setName(String name) {
        this.name = name;
    }
  
    public void setPurchasedPackage(String purchasedPackage) {
        this.purchasedPackage = purchasedPackage;
    }
}

During this blog post you will write the output data of your batch job to a CSV file. To be more specific, this CSV file must must fulfill these requirements:

  • It must use semicolon (‘;’) as a separator character.
  • Each row must contain the following information: the name of the student, the email address of the student, and the name of the purchased package.
  • The created CSV file must have a header row that contains the following the string: 'NAME;EMAIL_ADDRESS;PACKAGE'.

In other words, the CSV file that contains the processed student information must look as follows:

NAME;EMAIL_ADDRESS;PACKAGE
Tony Tester;tony.tester@gmail.com;master
Nick Newbie;nick.newbie@gmail.com;starter
Ian Intermediate;ian.intermediate@gmail.com;intermediate

Next, you will learn to add a header line to the created CSV file.

Adding a Header Line to the Created CSV File

When you want to add a header line to the created CSV file, you have to write a custom class which implements the FlatFileHeaderCallback interface. You can create this class by following these steps:

  1. Create a class that implements the FlatFileHeaderCallback interface.
  2. Add a private and final field called the header to the created class. This field contains the header that's written to the first line of the created CSV file.
  3. Create a constructor that injects the value of the header field by using constructor injection.
  4. Override the writeHeader(Writer writer) method of the FlatFileHeaderCallback interface.
  5. Write the header to the created CSV file by using the Writer object that's given to the writeHeader() method as a method parameter.

After you have implemented the FlatFileHeaderCallback interface, the source code of the StringHeaderWriter class looks as follows:

import org.springframework.batch.item.file.FlatFileHeaderCallback;

import java.io.IOException;
import java.io.Writer;

public class StringHeaderWriter implements FlatFileHeaderCallback {

    private final String header;

    StringHeaderWriter(String header) {
        this.header = header;
    }

    @Override
    public void writeHeader(Writer writer) throws IOException {
        writer.write(header);
    }
}
You don't have to:

  • Add a newline character to the header because it's added automatically by Spring Batch.
  • Flush the Writer.

Additional Reading:

Let's move on and find out how you can write the output data of your batch job a CSV file.

Writing Information to a CSV File

When you want to process the output data of your batch job, you have to configure an ItemWriter bean. Because you have to write the output data to a CSV File, you have to configure this bean by following these steps:

First, you have to create the configuration class that contains the beans which describe the workflow of your Spring Batch job. The source code of your configuration class looks as follows:

import org.springframework.context.annotation.Configuration;
  
@Configuration
public class SpringBatchExampleJobConfig {
}

Second, you have to create a FieldExtractor<T> object that extracts the field values from the object that is written to a CSV file. When you create a new FieldExtractor<T> object, you have to provide one type parameter which specifies the type of object that contains the input data of your ItemWriter.

You can create this object by following these steps:

  1. Add a private method to your configuration class and ensure that this method returns a FieldExtractor<StudentDTO> object.
  2. Ensure that this method returns a BeanWrapperFieldExtractor<StudentDTO> object. When you create a new BeanWrapperFieldExtractor<StudentDTO> object, you must specify the property names of the properties which are written to the created CSV file.

After you have implemented the method which returns a new FieldExtractor<T> object, the source code of your configuration class looks as follows:

import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor;
import org.springframework.batch.item.file.transform.FieldExtractor;
import org.springframework.context.annotation.Configuration;

@Configuration
public class SpringBatchExampleJobConfig {

    private FieldExtractor<StudentDTO> createStudentFieldExtractor() {
        BeanWrapperFieldExtractor<StudentDTO> extractor =
                new BeanWrapperFieldExtractor<>();
        extractor.setNames(new String[] {
                "name", 
                "emailAddress", 
                "purchasedPackage"
        });
        return extractor;
    }
}

Third, you have to create a new LineAggregator<T> object that creates the line which is written to the target file. When you create a new LineAggregator<T> object, you have to provide one type parameter that specifies the type of object that contains the input data of your ItemWriter.

When you want to create this object, you should add a new private method to your configuration class and ensure that this method returns a LineAggregator<StudentDTO> object. After you have added this method to your configuration class, you should implement it by following these steps:

  1. Create a new DelimitedLineAggregator<StudentDTO> object. This object transforms the input object into a String object that contains the property values provided the configured FieldExtractor<StudentDTO> object. These property values are separated by the specified delimiter character.
  2. Configure the used delimiter character (';').
  3. Ensure that the created DelimitedLineAggregator<StudentDTO> object object uses the FieldExtractor<StudentDTO> object which is returned by the createStudentFieldExtractor() method.
  4. Return the created DelimitedLineAggregator<StudentDTO> object.

After you have written the method which creates a new LineAggregator<StudentDTO> object, the source code of your configuration class looks as follows:

import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor;
import org.springframework.batch.item.file.transform.DelimitedLineAggregator;
import org.springframework.batch.item.file.transform.FieldExtractor;
import org.springframework.batch.item.file.transform.LineAggregator;
import org.springframework.context.annotation.Configuration;

@Configuration
public class SpringBatchExampleJobConfig {

    private LineAggregator<StudentDTO> createStudentLineAggregator() {
        DelimitedLineAggregator<StudentDTO> lineAggregator =
                new DelimitedLineAggregator<>();
        lineAggregator.setDelimiter(";");

        FieldExtractor<StudentDTO> fieldExtractor = createStudentFieldExtractor();
        lineAggregator.setFieldExtractor(fieldExtractor);

        return lineAggregator;
    }

    private FieldExtractor<StudentDTO> createStudentFieldExtractor() {
        BeanWrapperFieldExtractor<StudentDTO> extractor =
                new BeanWrapperFieldExtractor<>();
        extractor.setNames(new String[] {
                "name", 
                "emailAddress", 
                "purchasedPackage"
        });
        return extractor;
    }
}

Fourth, you have to create the method that configures your ItemWriter bean. Ensure that the created method takes an Environment object as a method parameter and returns an ItemWriter<StudentDTO> object.

After you have added this method to your configuration class, its source code looks as follows:

import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor;
import org.springframework.batch.item.file.transform.DelimitedLineAggregator;
import org.springframework.batch.item.file.transform.FieldExtractor;
import org.springframework.batch.item.file.transform.LineAggregator;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;

@Configuration
public class SpringBatchExampleJobConfig {

    @Bean
    public ItemWriter<StudentDTO> itemWriter(Environment environment) {

    }

    private LineAggregator<StudentDTO> createStudentLineAggregator() {
        DelimitedLineAggregator<StudentDTO> lineAggregator = 
                new DelimitedLineAggregator<>();
        lineAggregator.setDelimiter(";");

        FieldExtractor<StudentDTO> fieldExtractor = createStudentFieldExtractor();
        lineAggregator.setFieldExtractor(fieldExtractor);

        return lineAggregator;
    }

    private FieldExtractor<StudentDTO> createStudentFieldExtractor() {
        BeanWrapperFieldExtractor<StudentDTO> extractor 
                = new BeanWrapperFieldExtractor<>();
        extractor.setNames(new String[] {
                "name", 
                "emailAddress", 
                "purchasedPackage"
        });
        return extractor;
    }
}

Fifth, you have to implement the itemWriter() method by following these steps:

  1. Create a new FlatFileItemWriterBuilder<StudentDTO> object. This object creates FlatFileItemWriter<StudentDTO> objects which can write the output data of your batch job to a CSV file.
  2. Configure the name of the ItemWriter.
  3. Ensure that the created FlatFileItemWriter<StudentDTO> object writes a header line to the created CSV file by using the StringHeaderWriter class. You can read the header line from a properties file by using the Environment object given as a method parameter.
  4. Configure the LineAggregator object which constructs the lines which are written to the created CSV file.
  5. Configure the file path of the created CSV file. You can read this information from a properties file by using the Environment object given as a method parameter.
  6. Create a new FlatFileItemWriter<StudentDTO> object and return the created object.

After you have implemented the itemWriter() method, the source code of your configuration class looks as follows:

import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.file.builder.FlatFileItemWriterBuilder;
import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor;
import org.springframework.batch.item.file.transform.DelimitedLineAggregator;
import org.springframework.batch.item.file.transform.FieldExtractor;
import org.springframework.batch.item.file.transform.LineAggregator;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;

@Configuration
public class SpringBatchExampleJobConfig {

    @Bean
    public ItemWriter<StudentDTO> itemWriter(Environment environment) {
        String exportFilePath = environment
                .getRequiredProperty("batch.job.export.file.path");
        Resource exportFileResource = new FileSystemResource(exportFilePath);

        String exportFileHeader =
                environment.getRequiredProperty("batch.job.export.file.header");
        StringHeaderWriter headerWriter = new StringHeaderWriter(exportFileHeader);

        LineAggregator<StudentDTO> lineAggregator = createStudentLineAggregator();

        return new FlatFileItemWriterBuilder<StudentDTO>()
                .name("studentWriter")
                .headerCallback(headerWriter)
                .lineAggregator(lineAggregator)
                .resource(exportFileResource)
                .build();
    }

    private LineAggregator<StudentDTO> createStudentLineAggregator() {
        DelimitedLineAggregator<StudentDTO> lineAggregator
                = new DelimitedLineAggregator<>();
        lineAggregator.setDelimiter(";");

        FieldExtractor<StudentDTO> fieldExtractor
                = createStudentFieldExtractor();
        lineAggregator.setFieldExtractor(fieldExtractor);

        return lineAggregator;
    }

    private FieldExtractor<StudentDTO> createStudentFieldExtractor() {
        BeanWrapperFieldExtractor<StudentDTO> extractor = 
                new BeanWrapperFieldExtractor<>();
        extractor.setNames(new String[] {
                "name",
                "emailAddress",
                "purchasedPackage"
        });
        return extractor;
    }
}

You can now configure an ItemWriter bean which writes the output data of your batch job to a CSV file. Let’s summarize what you learned from this blog post.

Summary

This blog post has taught you four things:

  • If you have to write the output data of your batch job to a CSV file, you must use the FlatFileItemWriter<T> class.
  • If you have to add a header line to the created CSV file, you have to implement the FlatFileHeaderCallback interface.
  • The FlatFileItemWriter<T> class transforms the input objects into lines which are written to the CSV file by using a LineAggregator<T> object.
  • The DelimitedLineAggregator<T> class extracts the property values from the processed object by using a FieldExtractor<T> object.

The next part of this tutorial describes how you can write the output data of your batch job to an XML file.

P.S. You can get the example application of this blog post from Github.

17 comments… add one
  • Klaus UNGER Apr 16, 2016 @ 2:40

    thx for the good writeup.

    Spotted a minor copy/paste in DatabaseToXmlFileJobConfig code snippet.
    The outputFile is named students.csv instead of I guess intended students.xml

    • Petri Apr 16, 2016 @ 10:08

      Hi Klaus,

      Thank you for reporting this. It was indeed a copy-paste problem :P Anyway, it is fixed now.

    • Suresh Oct 10, 2016 @ 14:55

      Hi Petri,

      How to run the above code which si inside Github(example xml to database )

      Plse can u help this

      Thanks

      • Petri Oct 11, 2016 @ 19:53

        Hi,

        Take a look at the README.md files of the example applications. These files explain how you can run the Spring and Spring Boot examples. Remember that at the moment there is only ItemWriter that writes data to database by using JDBC. I will add more examples when I have time to write them.

  • kumaresan Sep 2, 2017 @ 7:53
    
    public class StudentDTO {
     
        private List emailAddress;
        private String name;
        private List purchasedPackage;
    
        //Getters and setters are omitted
    }
    
    

    my DTO Containing list of Address and List of Packages mean How cani write to CSV file Can you help me

    replay this mail id any one finds Solution of this

    I removed the email address from this comment because it revealed your employee - Petri

    • Petri Sep 2, 2017 @ 15:42

      Hi,

      Unfortunately it's impossible to propose a solution because you didn't describe the requirements of your CSV file.

      Also, since one person can have multiple addresses and packages, you have to either write multiple rows per person, or transform addresses and packages into strings that are written to the columns of your CSV file.

      After you have decided how you will solve this problem, you can configure the FlatFileItemWriter bean, and create the required LineAggregator and FieldExtractor objects that help you to extract the input data of your step and create a row that is written to the CSV file.

  • Anonymous Sep 15, 2017 @ 13:19

    Hi Petri,

    While trying to implement writing to XML part as stated above, I'm getting The method setClassesToBeBound(Class[]) in the type Jaxb2Marshaller is not applicable for the arguments (Class). Can you please let me know a way out.

    • Petri Sep 16, 2017 @ 0:00

      Hi,

      I am not sure what is going on because that particular method is using the varags notation and you should be able to use only one method parameter. Do you get the same error when you try to compile your code by using an IDE and Maven / Gradle?

  • Sid Sep 5, 2018 @ 7:40

    Do we need to pre - create the xml file before executing the job itself or is it created automatically?

    • Petri Sep 5, 2018 @ 8:48

      Hi,

      You don't have to create the target file. It should be created automatically by Spring Batch.

  • Amit Banger Dec 19, 2019 @ 11:08

    Is there a way to write own values to header?
    Let's suppose my header looks like {S_NO;Super_MID;MID;TID;Bank_Name;Rsp_code} and now values for columns {S_NO;Super_MID;MID;TID;Bank_Name} I can write through a file.
    But I want to set my own value for column {Rsp_code}, How to do this?
    I tried few things, didn't get it working.

  • Bandita Pradhan Aug 31, 2020 @ 21:25

    I have BillerOrder object which contains billerOrderId and OrderResponse[].
    @Getter
    @Setter
    @ToString
    public class BillerOrder {
    private String billerOrderId;
    private OrderResponse[] orderResponses;
    }

    How can I write to a flat file ? I am not able to figure out how to write OrderResponse[]

    • Petri Sep 10, 2020 @ 17:19

      Hi,

      It's not possible to write that object as is by using Spring Batch because a BilledOrder object can contain multiple OrderResponse objects. Before you can write that information to a flat file, you have to ensure that a billed order can have only one response. One way to do this is to create a new class (FlatBilledOrder) which has one billedOrderId and one OrderResponse object. After you have created this class, you have to ensure that your ItemReader returns FlatBilledOrder objects.

  • Arthur Oct 11, 2020 @ 11:27

    Thank you so much for share your knowledge

  • Vasu Nov 23, 2022 @ 15:09

    hi Petri,
    I have a situation where the value contains COMMA and my delimiter is also COMMA, so this row has an extra column. How can we fix it?

    • Petri Nov 29, 2022 @ 19:31

      Hi,

      The easiest solution is to change the delimiter character. If you cannot do this, you probably have to implement a custom FieldExtractor which changes the value if it's equal to your delimiter character.

Leave a Reply