I released five new sample lessons from my Test With Spring course: Introduction to Spock Framework

Spring Batch Tutorial: Writing Information to a File

My Spring Batch tutorial has taught us to read the input data of our batch job from different data sources. However, our batch jobs aren’t very useful because we don’t know how to save the data that is read from these data sources.

It’s time to take the next step and learn how we can save the data that has been processed by our Spring Batch job. This blog post helps us to write the processed data to CSV and XML files.

Let’s start by taking a quick look at our example application.

If you are not familiar with Spring Batch or Gradle, you should read the following blog posts before you continue reading this blog post:

Introduction to Our Example Application

During this tutorial we will implement several Spring Batch jobs that processes the student information of an online course. This time we will create Spring Batch jobs that write the processed student information to CSV and XML files. These files must contain a student list that provides the following information from each student:

  • The name of the student.
  • The email address of the student.
  • The name of the purchased package.

Before we can write this information to a file, we have provide the input data for the component that writes it to a file. In this case, this information is provided by using StudentDTO objects.

The StudentDTO class contains the information of a single student, and its source code looks as follows:

public class StudentDTO {

    private String emailAddress;
    private String name;
    private String purchasedPackage;

    public StudentDTO() {}

    public String getEmailAddress() {
        return emailAddress;
    }

    public String getName() {
        return name;
    }

    public String getPurchasedPackage() {
        return purchasedPackage;
    }

    public void setEmailAddress(String emailAddress) {
        this.emailAddress = emailAddress;
    }

    public void setName(String name) {
        this.name = name;
    }

    public void setPurchasedPackage(String purchasedPackage) {
        this.purchasedPackage = purchasedPackage;
    }
}

Let’s move on and find out how we can write information to a CSV file.

Writing Information to a CSV File

We have to create a CSV file that fulfills the following requirements:

  • It uses the semicolon (‘;’) as a separator character.
  • Each row must contain the following information: the name of the student, the email address of the student, and the name of the purchased package.
  • The created CSV file must have a header row that contains the following the string: ‘NAME;EMAIL_ADDRESS;PACKAGE’.

In other words, the CSV file that contains the processed student information must look as follows:

NAME;EMAIL_ADDRESS;PACKAGE
Tony Tester;tony.tester@gmail.com;master
Nick Newbie;nick.newbie@gmail.com;starter
Ian Intermediate;ian.intermediate@gmail.com;intermediate

We can configure the ItemWriter bean, which fulfills our requirements, by following these steps:

First, because we need add a header row into our CSV file, we have to create a component that writes the header to the created CSV file. We can do this by following these steps:

  1. Create a class that implements the FlatFileHeaderCallback interface.
  2. Add a private and final header field into the created class. This field contains the static header row that is written to the first line of the created CSV file.
  3. Create a constructor that injects the value of the header field by using constructor injection.
  4. Implement the writeHeader(Writer writer) method of the FlatFileHeaderCallback interface.
  5. Write the header to the created CSV file by using the Writer object that is given as a method parameter.

The source code of the StringHeaderWriter class looks as follows:

import org.springframework.batch.item.file.FlatFileHeaderCallback;

import java.io.IOException;
import java.io.Writer;

class StringHeaderWriter implements FlatFileHeaderCallback {

    private final String header;

    StringHeaderWriter(String header) {
        this.header = header;
    }

    @Override
    public void writeHeader(Writer writer) throws IOException {
        writer.write(header);
    }
}
It is important understand that we don’t have to:

  • Add the newline character into the header because it is added automatically by Spring Batch.
  • Flush the Writer.

Additional Reading:

Second, we have to create the configuration class that contains the beans which describe the workflow of our Spring Batch job. We can do this by creating a DatabaseToCsvFileJobConfig class and annotating it with the @Configuration annotation.

The source code of the DatabaseToCsvFileJobConfig class looks as follows:

import org.springframework.context.annotation.Configuration;

@Configuration
public class DatabaseToCsvFileJobConfig {
}

Third, we have to create a FieldExtractor<T> object that extracts the field values from the object that is written to a CSV file. When we create a new FieldExtractor<T> object, we have to provide one type parameter (T) which specifies the type of object that contains the input data of our ItemWriter.

Let’s start by adding a private createStudentFieldExtractor() method into the configuration class. Because the type of our input object is StudentDTO, this method must return a FieldExtractor<StudentDTO> object.

After we have added this method into our configuration class, we have to implement it by following these steps:

  1. Create a new BeanWrapperFieldExtractor<StudentDTO> object. This object extracts the field values from a StudentDTO object.
  2. Configure the names of the properties whose values are extracted. The order of the provided property names specify the order that is used when the found property values are written to a row of a CSV file. That is why we have to use this order: name, emailAddress, and purchasedPackage.
  3. Return the created object.

The source code of our configuration class looks as follows:

import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor;
import org.springframework.batch.item.file.transform.FieldExtractor;
import org.springframework.context.annotation.Configuration;

@Configuration
public class DatabaseToCsvFileJobConfig {

    private FieldExtractor<StudentDTO> createStudentFieldExtractor() {
        BeanWrapperFieldExtractor<StudentDTO> extractor = new BeanWrapperFieldExtractor<>();
        extractor.setNames(new String[] {"name", "emailAddress", "purchasedPackage"});
        return extractor;
    }
}

Fourth, we have to create a LineAggregator<T> object that is responsible of creating the line that is written to the target file. When we create a new LineAggregator<T> object, we have to provide one type parameter (T) that specifies the type of the input object.

Let’s start by adding a private createStudentLineAggregator() method to the configuration class. Because the type of our input object is StudentDTO, this method must return a LineAggregator<StudentDTO> object.

After we have added this method into our configuration class, we have to implement it by following these steps:

  1. Create a new DelimitedLineAggregator<StudentDTO> object. This object converts the input object into a String object that contains the property values provided a FieldExtractor<StudentDTO> object. These property values are delimited by using a delimiter character.
  2. Ensure that the created LineAggregator uses semicolon (‘;’) as a delimiter character.
  3. Configure the created LineAggregator to fetch the property values by using the FieldExtractor<StudentDTO> object that we created in step three.
  4. Return the created object.

The source code of our configuration class looks as follows:

import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor;
import org.springframework.batch.item.file.transform.DelimitedLineAggregator;
import org.springframework.batch.item.file.transform.FieldExtractor;
import org.springframework.batch.item.file.transform.LineAggregator;
import org.springframework.context.annotation.Configuration;

@Configuration
public class DatabaseToCsvFileJobConfig {

    private LineAggregator<StudentDTO> createStudentLineAggregator() {
        DelimitedLineAggregator<StudentDTO> lineAggregator = new DelimitedLineAggregator<>();
        lineAggregator.setDelimiter(";");

        FieldExtractor<StudentDTO> fieldExtractor = createStudentFieldExtractor();
        lineAggregator.setFieldExtractor(fieldExtractor);

        return lineAggregator;
    }

    private FieldExtractor<StudentDTO> createStudentFieldExtractor() {
        BeanWrapperFieldExtractor<StudentDTO> extractor = new BeanWrapperFieldExtractor<>();
        extractor.setNames(new String[] {"name", "emailAddress", "purchasedPackage"});
        return extractor;
    }
}

Fifth, we can finally create the ItemWriter<T> object that is responsible of writing the processed data to a CSV file. When we create a new ItemWriter<T> object, we have to provide one type parameter (T) that specifies the type of the input object.

Let’s start by adding a databaseCsvItemWriter() method to the DatabaseToCsvFileJobConfig class and annotating the created method with the @Bean annotation. Because the type of the input object is StudentDTO, this method must return an ItemWriter<StudentDTO> object.

After we have added this method into our configuration class, we have to implement it by following these steps:

  1. Create a new FlatFileItemWriter<StudentDTO> object. This object writes the processed data to a file.
  2. Ensure that the header: ‘NAME;EMAIL_ADDRESS;PACKAGE’ is written to the first line of the created file.
  3. Configure the file path of the created CSV file. Our example uses the path: ‘/tmp/students.csv’.
  4. Ensure that the created ItemWriter constructs the lines of the CSV file by using the LineAggregator<StudentDTO> object that we created in step four.
  5. Return the created object.

The source code of our configuration class looks as follows:

import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.file.FlatFileItemWriter;
import org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor;
import org.springframework.batch.item.file.transform.DelimitedLineAggregator;
import org.springframework.batch.item.file.transform.FieldExtractor;
import org.springframework.batch.item.file.transform.LineAggregator;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.FileSystemResource;

@Configuration
public class DatabaseToCsvFileJobConfig {

    @Bean
    ItemWriter<StudentDTO> databaseCsvItemWriter() {
        FlatFileItemWriter<StudentDTO> csvFileWriter = new FlatFileItemWriter<>();

        String exportFileHeader = "NAME;EMAIL_ADDRESS;PACKAGE";
        StringHeaderWriter headerWriter = new StringHeaderWriter(exportFileHeader);
        csvFileWriter.setHeaderCallback(headerWriter);

        String exportFilePath = "/tmp/students.csv";
        csvFileWriter.setResource(new FileSystemResource(exportFilePath));

        LineAggregator<StudentDTO> lineAggregator = createStudentLineAggregator();
        csvFileWriter.setLineAggregator(lineAggregator);

        return csvFileWriter;
    }

    private LineAggregator<StudentDTO> createStudentLineAggregator() {
        DelimitedLineAggregator<StudentDTO> lineAggregator = new DelimitedLineAggregator<>();
        lineAggregator.setDelimiter(";");

        FieldExtractor<StudentDTO> fieldExtractor = createStudentFieldExtractor();
        lineAggregator.setFieldExtractor(fieldExtractor);

        return lineAggregator;
    }

    private FieldExtractor<StudentDTO> createStudentFieldExtractor() {
        BeanWrapperFieldExtractor<StudentDTO> extractor = new BeanWrapperFieldExtractor<>();
        extractor.setNames(new String[] {"name", "emailAddress", "purchasedPackage"});
        return extractor;
    }
}

We have now created an ItemWriter bean that writes the processed student information into a CSV file. Let’s find out how we can write the same information to an XML file.

Writing Information to an XML File

We have to create an XML file that fulfills the following requirements:

  • The name of the root element must be: students.
  • The student element must contain the information of a single student.
  • The name element contains the name of the student.
  • The emailAddress element contains the email address of the student.
  • The purchasedPackage element contains the name of the of the purchased package.

In other words, the XML file that contains the processed student information looks as follows:

<students>
    <student>
        <name>Tony Tester</name>
        <emailAddress>tony.tester@gmail.com</emailAddress>
        <purchasedPackage>master</purchasedPackage>
    </student>
    <student>
        <name>Nick Newbie</name>
        <emailAddress>nick.newbie@gmail.com</emailAddress>
        <purchasedPackage>starter</purchasedPackage>
    </student>
    <student>
        <name>Ian Intermediate</name>
        <emailAddress>ian.intermediate@gmail.com</emailAddress>
        <purchasedPackage>intermediate</purchasedPackage>
    </student>
</students>

We can configure the ItemWriter bean, which fulfills our requirements, by following these steps:

First, we have to create the configuration class that contains the beans which describe the workflow of our Spring Batch job. We can do this by creating a DatabaseToXmlFileJobConfig class and annotating it with the @Configuration annotation.

The source code of the DatabaseToXmlFileJobConfig class looks as follows:

import org.springframework.context.annotation.Configuration;

@Configuration
public class DatabaseToXmlFileJobConfig {
}

Second, we have to create the ItemWriter<T> object that is responsible of writing the processed data to an XML file. When we create a new ItemWriter<T> object, we have to provide one type parameter (T) that specifies the type of the input object.

Let’s start by adding a databaseXmlItemWriter() method to the DatabaseToXmlFileJobConfig class and annotating the created method with the @Bean annotation. Because the type of the input object is StudentDTO, this method must return an ItemWriter<StudentDTO> object.

After we have added this method into our configuration class, we have to implement it by following these steps:

  1. Create a new StaxEventItemWriter<StudentDTO> object. This object writes the processed student information to an XML file by using StAX.
  2. Configure the file path of the created XML file. Our example uses the path: ‘/tmp/students.xml’.
  3. Configure the name of the root element. Our example uses the name: students.
  4. Ensure that the writer transforms the processed StudentDTO object into an XML fragment by using JAXB2.
  5. Return the created object.

The source code of the DatabaseToXmlFileJobConfig class looks as follows:

import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.xml.StaxEventItemWriter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.FileSystemResource;
import org.springframework.oxm.jaxb.Jaxb2Marshaller;

@Configuration
public class DatabaseToXmlFileJobConfig {

    @Bean
    ItemWriter<StudentDTO> databaseXmlItemWriter() {
        StaxEventItemWriter<StudentDTO> xmlFileWriter = new StaxEventItemWriter<>();

        String exportFilePath = "/tmp/students.xml";
        xmlFileWriter.setResource(new FileSystemResource(exportFilePath));

        xmlFileWriter.setRootTagName("students");

        Jaxb2Marshaller studentMarshaller = new Jaxb2Marshaller();
        studentMarshaller.setClassesToBeBound(StudentDTO.class);
        xmlFileWriter.setMarshaller(studentMarshaller);

        return xmlFileWriter;
    }
}

Third, before we can transform a StudentDTO object into an XML fragment that is written to the created XML file, we have to configure the name of the XML fragment’s root element.

We can do this by following these steps:

  1. Annotate the TodoDTO class with the @XmlRootElement annotation.
  2. Configure the name of the root element by setting the value of the @XmlRootElement annotation’s name attribute to: student.

The source code of the TodoDTO class looks as follows:

import javax.xml.bind.annotation.XmlRootElement;
 
@XmlRootElement(name="student")
public class StudentDTO {
 
    private String emailAddress;
    private String name;
    private String purchasedPackage;
 
    public StudentDTO() {}
 
    public String getEmailAddress() {
        return emailAddress;
    }
 
    public String getName() {
        return name;
    }
 
    public String getPurchasedPackage() {
        return purchasedPackage;
    }
 
    public void setEmailAddress(String emailAddress) {
        this.emailAddress = emailAddress;
    }
 
    public void setName(String name) {
        this.name = name;
    }
 
    public void setPurchasedPackage(String purchasedPackage) {
        this.purchasedPackage = purchasedPackage;
    }
}
Because the element names of the exported XML fragments are the same as the field names of the StudentDTO class, we don’t have to add additional annotations into the StudentDTO class. If this is not the case, or we need to use custom marshalling, we have to annotate our DTO class with JAXB annotations.

Additional Reading:

We have now created an ItemWriter bean that writes the processed student information into an XML file. It’s time to summarize what we learned from this blog post.

Summary

This blog post has taught us five things:

  • If we need to write information to a CSV file, we have to use the FlatFileItemWriter<T> class.
  • If we have to add a header row into the created CSV file, have to to implement the FlatFileHeaderCallback interface.
  • The FlatFileItemWriter<T> class transforms an object into a line by using a LineAggregator<T>.
  • If we need to write information to an XML file, we have to use the StaxEventItemWriter<T> class.
  • The StaxEventItemWriter<T> class transforms objects into XML fragments by using a Marshaller.

P.S. You can get the example applications of this blog post from Github: Spring example and Spring Boot example.

About the Author

Petri Kainulainen is passionate about software development and continuous improvement. He is specialized in software development with the Spring Framework and is the author of Spring Data book.

About Petri Kainulainen →

5 comments… add one
  • thx for the good writeup.

    Spotted a minor copy/paste in DatabaseToXmlFileJobConfig code snippet.
    The outputFile is named students.csv instead of I guess intended students.xml

    Reply
    • Hi Klaus,

      Thank you for reporting this. It was indeed a copy-paste problem :P Anyway, it is fixed now.

      Reply
    • Hi Petri,

      How to run the above code which si inside Github(example xml to database )

      Plse can u help this

      Thanks

      Reply
      • Hi,

        Take a look at the README.md files of the example applications. These files explain how you can run the Spring and Spring Boot examples. Remember that at the moment there is only ItemWriter that writes data to database by using JDBC. I will add more examples when I have time to write them.

        Reply

Leave a Comment