Example using Spring Integration with Spring Batch as a shell script alternative for a single deployable jar with all code to run a batch process application. Using Spring Integration In Conjunction With Spring Batch. Mark Fricke July 10. Recently I was working on a development project for a client focused on Spring Batch.
Photo credit: Spring Source
Spring Batch, is an open source framework for batch processing – execution of a series of jobs. Spring Batch provides classes and APIs to read/write resources, transaction management, job processing statistics, job restart and partitioning techniques to process high-volume of data.
The following Spring Batch tutorials and examples are tested with :
- Spring Core 3.2.2.RELEASE
- Spring Batch 2.2.0.RELEASE
P.S Spring Batch is part of the Spring Portfolio.
1. Quick Guide
A simple batch job, and run with command line.
- Spring Batch Hello World Example
Short description about Spring batch, and create a job to read data from a CSV File, process it, and write it to an XML file (JAXB). - Spring Batch TaskletStep example
Example to cleanup the resources after the batch job has completed. - Run Spring Batch Job With CommandLineJobRunner
Run Spring batch job with command line.
2. ItemReader, ItemProcessor, ItemWriter
Few examples to show the use of Spring batch classes to read/write resources (csv, xml and database).
- Spring Batch Example – CSV File To MySQL database
Read data from a CSV file and write it into a MySQL database, job meta is stored in the database. - Spring Batch Example – XML File To MongoDB database
Read data from a XML file (XStream) and write it into a nosql database MongoDB, also unit test the batch job. - Spring Batch Example – XML File To CSV File
Read data from a XML file (JAXB2), process with ItemProcessor and write it into a CSV file. It also shows you how to convert Date and BigDecimal data type to/from object via JAXB2. - Spring Batch Example – MySQL Database To XML
Read data from a MySQL database and write it into an XML file (XStream), also the use of jobParameters. - Spring Batch MultiResourceItemReader example
A ItemReader to read multiple files.
3. Scheduler
Run batch job with scheduler frameworks.
- Spring Batch + Spring TaskScheduler example
Spring TaskScheduler to schedule a batch job to run every 5 seconds. - Spring Batch + Quartz Scheduler example
Integrate Quartz to schedule a batch job to run every 10 seconds.
4. Unit Test
- Spring Batch unit test example
How to unit test a batch job or individual steps.
5. Advance
- Spring Batch partitioner example
Multiple threads to process range of data. - Spring Batch listener example
Examples to intercept the step execution of a batch job.
6. Spring Batch FAQs
Some common questions and answers in Spring Batch.
References
About the Author
mkyong
Founder of Mkyong.com, love Java and open source stuff. Follow him on Twitter. If you like my tutorials, consider make a donation to these charities.Comments
This is an example program for Spring Batch
This is the test of spring batch which read from db table and write to db table (mysql)
By default spring batch DataSource uses the HSQL. Here I am using JPA and set the database platform (like MySQL) to point to a specific database (other than the default HSQL).
Used Following
- spring-batch-core 3.0.1.RELEASE
- spring-boot-starter-batch 1.1.5.RELEASE
- spring-boot-starter-data-jpa 1.1.5.RELEASE
- mysql
- gradle
I am also using gradle wrapper (http://www.gradle.org/docs/current/userguide/gradle_wrapper.html). The wrapper is something you should check into version control. By distributing the wrapper with your project, anyone can work with it without needing to install Gradle beforehand.Add following to build.gradle
- task wrapper(type: Wrapper) {gradleVersion = '1.11'}
Command to create gradlew:-
- gradle wrapper
I am also using Spring Boot (http://projects.spring.io/spring-boot/). This helps to create a stand-along spring app - just run the app. To make the app spring-boot add the following line to build.gradle
- apply plugin: 'spring-boot'
- following Dependencies:-
** compile 'org.springframework.boot:spring-boot-starter-batch:1.1.5.RELEASE'
** compile 'org.springframework.boot:spring-boot-starter-data-jpa:1.1.5.RELEASE'
Command to spring boot
- gradlew bootRun
- gradlew run
DDL:-
CREATE TABLE
person
(id
bigint(20) NOT NULL AUTO_INCREMENT,first_name
varchar(45) DEFAULT NULL,last_name
varchar(45) DEFAULT NULL,PRIMARY KEY (id
)) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8;CREATE TABLE
account
(accountId
bigint(20) NOT NULL,firstName
varchar(45) DEFAULT NULL,lastName
varchar(45) DEFAULT NULL,PRIMARY KEY (accountId
)) ENGINE=InnoDB DEFAULT CHARSET=utf8;INSERT INTO
account
(accountId
,firstName
,lastName
)VALUES(1,'first name','last name');