Create a @JobScope @Service with Spring Batch that holds shared memory for a job?

Question

Essentially what i'm looking to do is create a @Service or component that loads some data into memory from a database table which is referenced throughout the job execution

package com.squareup.se.bridge.batchworker.components.context;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.common.base.Strings;
import com.squareup.se.bridge.batchworker.repositories.BridgeBatchJobParametersRepository;
import com.squareup.se.bridge.batchworker.util.JobParameterKeys;
import com.squareup.se.bridge.core.api.services.batchworker.FatalSyncException;
import com.squareup.se.bridge.core.integration.util.logger.JobExecutionLoggerFactory;
import java.io.IOException;
import javax.validation.constraints.NotNull;
import org.slf4j.Logger;
import org.springframework.batch.core.JobExecution;
import org.springframework.batch.core.JobExecutionListener;
import org.springframework.batch.core.configuration.annotation.JobScope;
import org.springframework.stereotype.Component;

@JobScope @Component public class BridgeBatchIntegrationJobContextProvider
    implements JobExecutionListener {
  private Logger logger;
  private ObjectMapper mapper;
  private BridgeBatchJobParametersRepository bridgeBatchJobParametersRepository;
  private BridgeIntegrationJobContext context;

  public BridgeBatchIntegrationJobContextProvider(ObjectMapper mapper,
      BridgeBatchJobParametersRepository bridgeBatchJobParametersRepository) {
    this.mapper = mapper;
    this.bridgeBatchJobParametersRepository = bridgeBatchJobParametersRepository;
  }

  @Override public void beforeJob(JobExecution jobExecution) {
    var jobId = jobExecution.getJobParameters().getString(JobParameterKeys.SYNC_ID);
    this.logger = JobExecutionLoggerFactory.getLogger(
        BridgeBatchIntegrationJobContextProvider.class, jobId);
    this.context = deserializeJobParameters(jobId);
  }

  @NotNull public BridgeIntegrationJobContext get() {
    if (context == null) {
      throw new IllegalStateException("Expected context to exist before calling this method");
    }
    return context;
  }

  @Override public void afterJob(JobExecution jobExecution) { }

  @NotNull private String getParameters(String jobId) {
    var jobParams = bridgeBatchJobParametersRepository.find(jobId);
    if (jobParams == null || jobParams.size() == 0) {
      throw new FatalSyncException(String.format("No job parameters for job `%s` exists", jobId));
    }
    if (jobParams.size() > 1) {
      throw new FatalSyncException(String.format("Multiple parameter entries exist for job `%s`",
          jobId));
    } else if (Strings.isNullOrEmpty(jobParams.get(0).getIntegrationContext())) {
      throw new FatalSyncException(String.format("Job parameters for job `%s` is empty", jobId));
    }
    return jobParams.get(0).getIntegrationContext();
  }

  @NotNull private BridgeIntegrationJobContext deserializeJobParameters(String jobId) {
    try {
      return mapper.readValue(getParameters(jobId),
          BridgeIntegrationJobContext.class);
    } catch (IOException e) {
      //TODO page on this
      logger.info(e.getMessage(), e);
      throw new FatalSyncException(e);
    }
  }
}

I've configured a job like this:

return jobBuilderFactory.get(CUSTOMERS_BATCH_JOB_NAME)
        .incrementer(new RunIdIncrementer())
        .start(loadFromOriginStep)
        .next(retryFailuresFromOriginStep)
        .listener(bridgeBatchIntegrationJobContextProvider)
        .listener(jobListener)
        .build();

The constructor depends on other beans including a jackson object mapper and a JPA repo. I'm encountering a few problems:

the constructor is not instantiated by Spring and thus the instance variables I want to bind are not present

If I remove @JobScope from the component, Spring constructs the component instance.

score 0 · Accepted Answer · answered May 11 '20 at 09:19

0

I don't see where @JobContext is used in your code, and according to your requirement, you don't need it.

If you want to load some data in the job execution context using a listener, you can do it in beforeJob with jobExecution.getExecutionContext().put("key", "value");.

That said, it is not recommended to load a lot of data in the execution context as it is persisted between steps.

So unless you are loading a small amount of data in the execution context, you need to find another approach (like using a separate cache for example, see Spring Batch With Annotation and Caching).

answered May 11 '20 at 09:19

Mahmoud Ben Hassine

28,519
3
32
50

Maybe I wasn't clear. I am aware of the limitations of the execution context, and this is exactly what I'm trying to workaround. I've created a table to store a large blob to hold my own execution context. I am attempting to make a bean which only exists for the lifetime of and is unique to the executing job which loads the data stored in the table before execution and deletes it after. I saw @JobScope annotation and I assumed this was relevant. – JSF May 11 '20 at 17:13
@Mahmoud Ben Hassine But you said that loading additional job parameters in beforeJob() is too late in 2018 here(https://stackoverflow.com/questions/53118641/adding-job-parameter-in-before-job-doesnt-work-spring-batch). Which one is right? – Wood Oct 25 '22 at 13:35
@Sam It's late for job parameters, but not for any business data that is needed by the steps of the job (like cache reference data for example). – Mahmoud Ben Hassine Oct 25 '22 at 14:28

Create a @JobScope @Service with Spring Batch that holds shared memory for a job?

1 Answers1