Spring Batch

SPRING BATCH

1. Introduction

Spring Batch is a lightweight, comprehensive batch framework which enables bulk processing of business Operations. 

A typical batch program generally reads a large number of records from a database, file, or queue, processes the data in some fashion, and then writes back data in a modified form.  

Spring Batch automates this basic batch iteration providing a robust, enterprise-scale solution to batch processing.  

2. Spring Batch Advantage:

ü Provides reusable functions that are essential in processing large volumes of records. 

ü Highly scalable, Easy-to-use & Customizable. 

ü Transaction Management by Spring API’s.  Optimized by committing periodically (chunk)

ü Exception handling Capability - Automatic retry after failure. RetryTemplate, Completion Policy to handle the exception. 

ü Multiple job execution Options - Task Executor, Partition Step, Remote Chunking approach (messagingGateway).

ü Other features includes processing statistics, job restart, skip, and resource management.

ü Spring Batch Admin provides a web-based user interface that features an admin console for Spring Batch applications and systems.

3. Getting Started (Spring-Batch-Core)

The following steps explain how to create a simple Spring Batch Core Module. 

Configure pom with Spring Batch dependencies. 

 <dependency>  
    <groupId>org.springframework.batch</groupId>  
    <artifactId>spring-batch-core</artifactId>  
 </dependency>   
 <dependency>    
    <groupId>org.springframework.batch</groupId> 
    <artifactId>spring-batch-infrastructure</artifactId>  
 </dependency>

3.1. Spring Batch configurations (xml) :

Configure spring batch jobs as below in spring-batch.xml 

<!-- BatchJob1 Configuration --> 

 <job id="batchJob1" job-repository="jobRepository" incrementer="incrementer" xmlns="http://www.springframework.org/schema/batch">
  <step id="step1" >   
  <tasklet transaction-manager="transactionManager" start-limit="100" >   
  <chunk reader="batchReader"  processor="batchProcessor"  writer="batchWriter"  commit-interval="4"/>                            
  </tasklet>
  </step> 
</job>

<bean id="incrementer" class="com.rainbow.batch.util.BatchJobParametersIncrementer"/>

3.2 Configure JobRepository(InMemory Job Repository) and JdbcCursoritemReader (using @Configuration annotation) :

@Bean 
 public AbstractJobRepositoryFactoryBean jobRepository() 
 {  
  AbstractJobRepositoryFactoryBean inMemJobRepository = new MapJobRepositoryFactoryBean();
  inMemJobRepository.setTransactionManager(jdbcTransactionManager());
  inMemJobRepository.setIsolationLevelForCreate("ISOLATION_DEFAULT");  
  return inMemJobRepository;   
 }

 @Bean
 public JdbcCursorItemReader jdbcCursorReader() 
{
  JdbcCursorItemReader jdbcCursorItemReader = new JdbcCursorItemReader();
  jdbcCursorItemReader.setDataSource(emiaDataSource());
  jdbcCursorItemReader.setSql(batchReader.SELECT_QUERY);
  jdbcCursorItemReader.setRowMapper(batchTORowMapper());
  return jdbcCursorItemReader;
}
 

3.3. Create Batch Reader :

/**  Read from Source Database using JdbcCursorItemReader and pass the result to Processor defined in batch-context. 

@Component("batchReader")
public class BatchReader implements ItemReader, StepExecutionListener 
{ 
 public static final String SELECT_QUERY = "select * FROM ABC Table ";

 @Override
 public BatchTO read() throws Exception {  
  Object cursor = jdbcCursorReader.read();  
  if ( cursor !=null ) {                
   batchTO = (BatchTO)cursor;   
   log.info("reader():batchTO.getBatch_Type() : " + batchTO getBatch_Type());         
   return batchTO;
  }
  else {
   return null; 
  }   
 } 

3.4. Create Batch Processor :

/** Process data received from Reader and pass the result to Writer.

@Component("batchProcessor")
public class BatchProcessor implements ItemProcessor {

 @Override
 public BatchTO process(BatchTO batchTO) throws Exception
 {
  batchTO.setBatch_Type("{" + batchTO.getBatch_Type() + "}");
  return batchTO      
 }  

3.5. Create Batch Writer :

@Component("batchWriter")
public class BatchWriter implements ItemWriter 
{ 
 
@Override
 public void write(List writerList) throws Exception {     
  for (Iterator iterator = (Iterator) writerList.iterator(); iterator.hasNext();) {
   BatchTO batchTO = iterator.next();
   log.info("write():batchTO.getBatch_Type() : " + batchTO.getBatch_Type());
  }
}

3.6. Trigger the job using JobLauncher  :

@Test
  public void testBatchJob1() throws Exception 
 {
     @Autowired
     private JobLauncher emiaJobLauncher;
 
               jobExec = batchJobLauncher.run(batchJob1, new JobParameters(parametersMap));
}

4. Spring-Batch-Admin :

The following steps explain how to create a simple Spring Batch Admin Module. 

Steps

4.1. Configure Maven pom.xml with Spring Batch Admin dependencies. 

 <dependency>    
   <groupId>org.springframework.batch</groupId>
   <artifactId>spring-batch-admin-manager</artifactId> 
 </dependency>  

 <dependency>    
   <groupId>org.springframework.batch</groupId> 
   <artifactId>spring-batch-admin-resources</artifactId>  
 </dependency>

4.2. Spring Batch configurations (xml) :

<!-- configure Transaction Manager and dataSource--> 

  <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
  <property name="dataSource" ref="dataSource" />
 </bean>
  
 <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">  
  <property name="driverClassName" value="${batch.jdbc.driver}" />
  <property name="url" value="${batch.jdbc.url}" />
  <property name="username" value="${batch.jdbc.user}" />
  <property name="password" value="${batch.jdbc.password}" />
 </bean>

 4.3. Configure JobRepository(JobRepositoryFactoryBean) and JdbcCursoritemReader (using @Configuration annotation) :

@Bean 
 public JobRepositoryFactoryBean jobRepository() 
 {       
  JobRepositoryFactoryBean jobRepositoryFactoryBean = new JobRepositoryFactoryBean();
  jobRepositoryFactoryBean.setTransactionManager(transactionManager);   
  jobRepositoryFactoryBean.setDatabaseType("mysql");
  jobRepositoryFactoryBean.setDataSource(dataSource);
  jobRepositoryFactoryBean.setTablePrefix("BATCH_");
  jobRepositoryFactoryBean.setIsolationLevelForCreate("ISOLATION_DEFAULT");
  return jobRepositoryFactoryBean;   
 }

4.4. Create below tables using schema-mysql-create.sql.

    

               

4.5. Build and deploy the war File. Launch the Batch Admin Console to trigger the Jobs. Jobs Screen list all available Jobs.  
  

            

             

4.6. Select the respective jobs, enter required parameters and launch the job. Status of the job can be tracked using Executions tab. Batch Admin Module uses the above created tables to store batch execution details.

  

           

            

5. References : 

http://static.springsource.org/spring-batch/reference/html/configureJob.html

                    Last modified @ 01-april-2013 by Shameer Thaha.