Spring Batch With Annotation and Caching

Question

Does anyone have good example of Spring Batch (Using Annotation) to cache a reference table which will be accessible to processor ?

I just need a simple cache, run a query which returns some byte[] and keep it in memory till the time job is executing.

Appreciate any help on this topic.

Thanks !

Mahmoud Ben Hassine · Accepted Answer · 2020-05-19T09:05:21.043

A JobExecutionListener can be used to populate the cache with reference data before the job is executed and clear the cache after the job is finished.

Here is an example:

import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.CacheManager;
import org.springframework.cache.concurrent.ConcurrentMapCacheManager;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableBatchProcessing
public class MyJob {

    private JobBuilderFactory jobs;

    private StepBuilderFactory steps;

    public MyJob(JobBuilderFactory jobs, StepBuilderFactory steps) {
       this.jobs = jobs;
       this.steps = steps;
    }

    @Bean
    public CacheManager cacheManager() {
        return new ConcurrentMapCacheManager(); // return the implementation you want
    }

    @Bean
    public Tasklet tasklet() {
        return new MyTasklet(cacheManager());
    }

    @Bean
    public Step step() {
        return steps.get("step")
                .tasklet(tasklet())
                .build();
    }

    @Bean
    public JobExecutionListener jobExecutionListener() {
        return new CachingJobExecutionListener(cacheManager());
    }

    @Bean
    public Job job() {
        return jobs.get("job")
                .start(step())
                .listener(jobExecutionListener())
                .build();
    }

    class MyTasklet implements Tasklet {

        private CacheManager cacheManager;

        public MyTasklet(CacheManager cacheManager) {
            this.cacheManager = cacheManager;
        }

        @Override
        public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
            String name = (String) cacheManager.getCache("referenceData").get("foo").get();
            System.out.println("Hello " + name);
            return RepeatStatus.FINISHED;
        }
    }

    class CachingJobExecutionListener implements JobExecutionListener {

        private CacheManager cacheManager;

        public CachingJobExecutionListener(CacheManager cacheManager) {
            this.cacheManager = cacheManager;
        }

        @Override
        public void beforeJob(JobExecution jobExecution) {
            // populate cache as needed. Can use a jdbcTemplate to query the db here and populate the cache
            cacheManager.getCache("referenceData").put("foo", "bar");
        }

        @Override
        public void afterJob(JobExecution jobExecution) {
            // clear cache when the job is finished
            cacheManager.getCache("referenceData").clear();
        }
    }

    public static void main(String[] args) throws Exception {
        ApplicationContext context = new AnnotationConfigApplicationContext(MyJob.class);
        JobLauncher jobLauncher = context.getBean(JobLauncher.class);
        Job job = context.getBean(Job.class);
        jobLauncher.run(job, new JobParameters());
    }

}

When executed, it prints:

Hello bar

which means data is correctly retrieved from the cache. You would need to adapt the sample to query the database and populate the cache (See comments in code).

Hope this helps.

Does it affect job restart ability if I override data in cache with required data for each step. — Prashant Shilimkar, May 02 '19 at 03:38
Is there any way we can apply multi threading read data from 4 Tables which each contains 10 lacs records and later I want to use it to embed into another object to create final Object ? Reading each table takes 10 mins, so I want to faster this process. Any guidance around this ? — , May 19 '20 at 10:33
Is there any way if we can do chunk based processing Cache Data? — PAA, Feb 24 '22 at 07:56

score 3 · Answer 2 · answered Oct 04 '18 at 10:31

3

You can use ehcache-jsr107 implementation. Very quick to setup. Spring and ehcache integration example is available here. You should be able to setup same with spring batch also.

Hope this hleps

answered Oct 04 '18 at 10:31

Niraj Sonawane

10,225
10
75
104

Spring Batch With Annotation and Caching

2 Answers2

Linked