Are you tired of writing tests which have a lot of boilerplate code? If so, get started with Spock Framework >>

Spring Batch Tutorial: Getting the Required Dependencies With Maven

The first part of my Spring Batch tutorial explained why we should use Spring Batch instead of writing our own batch jobs and identified the basic building blocks of a Spring Batch job. Before we can move on and see Spring Batch in action, we have to get the required dependencies.

After we have read this blog post, we:

  • Can identify the modules provided by Spring Batch.
  • Can list the required dependencies when we are using Spring Framework or Spring Boot.
  • Know how we can get the required dependencies with Maven.

Let’s start by taking a look at the modules provided by Spring Batch.

This blog post assumes that:

Introduction to the Modules of Spring Batch

Spring Batch provides the following modules:

  • The spring-batch-infrastructure module contains the common readers and writers, and provides services for application developers and the core module.
  • The spring-batch-core module contains the classes which are required to launch and control Spring Batch jobs.
  • The spring-batch-test module provides support for writing automated tests for Spring Batch jobs.
  • The spring-batch-integration module helps us to integrate Spring Batch with Spring Integration.

Next, we will find out how we can get the required dependencies when we are writing a “legacy” Spring application.

Getting the Dependencies of a “Legacy” Spring Application

When we are writing Spring Batch jobs by using Spring Framework, we have to get the following dependencies:

  • Because Spring Batch writes the job and step execution metadata (aka logs) to the database, we need a JDBC driver which allows our application to communicate with the used database. We use the H2 in-memory database because it makes our example application easy to run. Also, we need this dependency only when our application is run, and that’s why we have to use the runtime scope when we declare this dependency.
  • Liquibase. We integrate Liquibase with Spring Framework and ensure that it creates the database tables which contain the job and step execution metadata when the Spring container is started.
  • The datasource provides database connections to our application. We use the HikariCP datasource because it’s the fastest datasource on this planet.
  • Spring Batch Core contains the classes that are required to launch and control Spring Batch jobs. Also, it includes the spring-batch-infrastructure module.
Spring applications require other dependencies as well. I didn’t include those dependencies in the previous list simply because every application is different and requires different dependencies.

When we are writing a “legacy” Spring application, we can get the required dependencies by using one of these two options:

  1. We can manage the dependency versions by using the dependency management of Spring Boot.
  2. We can use the traditional way and manage the dependency versions manually.

Let’s find out how we can get the required dependencies when we use the dependency management of Spring Boot.

Using the Dependency Management of Spring Boot

We can get the required dependencies by following these steps:

First, we have to configure the parent POM of our Maven project. Spring Boot has starter called spring-boot-starter-parent which provides dependency and plugin management for Maven projects. When we want to use the dependency management of Spring Boot, we have to set this starter as the parent of our Maven project. We can do this by adding the following XML to our POM file:

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.3.1.RELEASE</version>
</parent>

Second, after we have configured the parent POM of our Maven project, we don’t have to worry about dependency versions because they are inherited from the parent POM. This means that we can get the required dependencies by adding the following XML to the dependencies section of our POM file:

<!-- Database -->
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <scope>runtime</scope>
</dependency>

<!-- Liquibase -->
<dependency>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-core</artifactId>
</dependency>
        
<!-- DataSource -->
<dependency>
    <groupId>com.zaxxer</groupId>
    <artifactId>HikariCP</artifactId>
</dependency>

<!-- Spring Batch -->
<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-core</artifactId>
</dependency>

Next, we will find out how we can manage our dependency versions manually.

Using the Traditional Way

If we want to use the traditional way, we have to manage our dependency versions “manually”. In other words, we must specify the versions of all dependencies. We can do this by adding the following dependency declarations to the dependencies section of our pom.xml file:

<!-- Database -->
<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <version>1.4.200</version>
    <scope>runtime</scope>
</dependency>

<!-- Liquibase -->
<dependency>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-core</artifactId>
    <version>3.8.9</version>
</dependency>

<!-- DataSource -->
<dependency>
    <groupId>com.zaxxer</groupId>
    <artifactId>HikariCP</artifactId>
    <version>3.4.5</version>
</dependency>

<!-- Spring Batch -->
<dependency>
    <groupId>org.springframework.batch</groupId>
    <artifactId>spring-batch-core</artifactId>
    <version>4.2.4.RELEASE</version>
</dependency>

We can now get the required dependencies when we are working with a “legacy” Spring application. Let’s move on and find out how we can get the required dependencies when we are using Spring Boot.

Getting the Dependencies of a Spring Boot Application

We can get the required dependencies by following these steps:

First, we have to configure the parent POM of our Maven project. When we are writing a Spring Boot application, we must set the spring-boot-starter-parent as the parent of our Maven project. We can do this by adding the following XML to our POM file:

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.3.1.RELEASE</version>
</parent>

Second, we have to configure the required dependencies. When we configure these dependencies, we can ignore the dependency versions because they are inherited from the parent POM. Before we can use Spring Batch in a Spring Boot application, we must get the following dependencies:

  • Because Spring Batch writes the job and step execution metadata (aka logs) to the database, we need a JDBC driver which allows our application to communicate with the used database. We use the H2 in-memory database because it makes our example application easy to run. Also, we need this dependency only when our application is run, and that’s why we have to use the runtime scope when we declare this dependency.
  • Liquibase. We integrate Liquibase with Spring Boot and ensure that it creates the database tables which contain the job and step execution metadata when the Spring container is started. Because we need this dependency only when our application is run, we have to use the runtime scope when we declare this dependency.
  • The spring-boot-starter-batch dependency provides the dependencies which are required by Spring Batch.
  • The spring-boot-starter-jdbc dependency is a starter that provides the dependencies which allow us to use JDBC and HikariCP datasource in our Spring Boot application.
Spring Boot applications require other dependencies as well. I didn’t include those dependencies in the previous list simply because every application is different and requires different dependencies.

We can get these dependencies by adding the following dependency declarations to the dependencies section of our pom.xml file:

<dependency>
    <groupId>com.h2database</groupId>
    <artifactId>h2</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>org.liquibase</groupId>
    <artifactId>liquibase-core</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>

We can now get the required dependencies when we are using Spring Boot. Let’s summarize what we learned from this blog post.

Summary

This blog post has taught us five things:

  • Because Spring Batch writes the job and step execution metadata to a database, we need a JDBC driver which allows us to communicate with the used database.
  • Because we want to communicate with a relational database, we need a datasource which allows us to create database connections.
  • Because Spring Batch writes the job and step execution metadata to a database, we need a way to create the required database tables before Spring Batch tries to insert data into these tables. We can use Liquibase (or Flyway) for this purpose.
  • If we are working with a “legacy” Spring application, we have to declare the spring-batch-core dependency in our POM file.
  • If we are using Spring Boot, we have to declare the spring-boot-starter-batch dependency in our POM file.

The next part of this tutorial describes how we can get the required dependencies with Gradle.

P.S. You can get the example applications of this blog post from Github.

15 comments… add one
  • Thanks Petri for another insightful blog post.

    Looking forward to the rest of this new series – I remember trying to use Spring batch a couple of years back and found it very complicated. As I had a deadline ended up abandoning it. So am hoping to learn it finally though this blog series.

    I do find it odd that you are using JPA (Hibernate) for a batch process – I always preferred using JDBC directly for batch process. But am sure this is an opportunity for me to broaden my horizon and learn more :)

    I enjoyed your series on Spring Data which inspired me to buy your book.
    Am excited to see this where new series leads to.

    Best wishes,
    SGB

    Reply
    • Hi,

      Thank your for your kind words. I really appreciate them.

      I remember trying to use Spring batch a couple of years back and found it very complicated. As I had a deadline ended up abandoning it. So am hoping to learn it finally though this blog series.

      I have heard similar comments from my colleagues as well, and that is why I decided to write this tutorial.

      I do find it odd that you are using JPA (Hibernate) for a batch process – I always preferred using JDBC directly for batch process. But am sure this is an opportunity for me to broaden my horizon and learn more :)

      This is a good point. I have to confess that typically I use JDBC as well (for batch processes), but I decided to cover both JDBC and JPA because there are a few situations when it makes sense to use JPA. In other words, sometimes the benefits of using JPA are so high that it makes sense to take a performance hit.

      Reply
  • Hi

    The following dependency in your article , is wrong because till now only 1.4.191 , is latest version for h2 data base ,but you mentioned the version as , 1.4.90 , so can you please check

    com.h2database
    h2
    1.4.90

    Reply
    • Hi,

      If you specify your dependencies manually, you should naturally use the newest versions. However, if you use Spring IO Platform or Spring Boot, your best option is to not specify dependency versions and use the versions that are guaranteed to work (provided by parent or starter POMs).

      Reply
      • Hi

        I agree the automatically downloading dependencies using sping IO , am just telling you that you mentioned wrong version number for the dependency , as 1.4.90 , am just telling you it is 1.4.191 , please observe the numbers and correct in your article

        Reply
        • Hi,

          This post uses the dependency versions that provided by the Spring IO Platform 2.0.1.RELEASE because this way I can guarantee that they are working with each other. Because the Spring IO Platform 2.0.1.RELEASE uses H2 1.4.190, this article will use it as well.

          Reply
          • lol. Hilarious!
            I think Ramesh is saying that there is a typo in the section “Tradition Way”.
            For the h2 dependency, in your pom file, you have listed version 1.4.90 instead of 1.4.190.

            90 vs 190.

          • Oops. I fixed it. Thank you for pointing this out.

  • Hi Petri,
    I tried running your spring boot batch code (excelFileToDatabaseJob) and seeing the below exception
    Caused by: java.lang.IllegalArgumentException: Sheet index (1) is out of range (0..0)
    at org.apache.poi.xssf.usermodel.XSSFWorkbook.validateSheetIndex(XSSFWorkbook.java:1158) ~[poi-ooxml-3.11.jar:3.11]
    at org.apache.poi.xssf.usermodel.XSSFWorkbook.getSheetAt(XSSFWorkbook.java:934) ~[poi-ooxml-3.11.jar:3.11]
    at org.apache.poi.xssf.usermodel.XSSFWorkbook.getSheetAt(XSSFWorkbook.java:106) ~[poi-ooxml-3.11.jar:3.11]
    at org.springframework.batch.item.excel.poi.PoiItemReader.getSheet(PoiItemReader.java:47) ~[spring-batch-excel-0.5.0-SNAPSHOT.jar:na]

    I created a new project that reads an excel and processes it using spring-batch-excel code and seeing the same exception as well. Have you seen this happening?

    Reply
    • Hi Sharmila,

      I haven’t seen that exception. However, it looks like the spreadsheet has an empty row somewhere, and this is why the exception is thrown.

      Did you modify the spreadsheet or replace it with some other spreadsheet? Also, if you opened it with Excel, Excel might have modified it for you :| The reason why I ask this is that I cannot reproduce this with my version. :(

      Reply
  • Dear Sir,

    Thank you very much for the code. it saved me lot of time. I am executing from web application and passing path dynamically. If any one still looking for dynamic path, send me the mail. I will post the code here.

    With Regards
    Ramesh

    Reply
    • Thank you for kind words. I really appreciate them. Also, I will keep your offer in mind and ask additional information if someone needs it.

      Reply

Leave a Comment