A job instance already exists when scheduling through spring cloud dataflow server

Question

I have deployed the Custom built SCDF dataflow-server-25x in openshift and I'm trying to run my Spring boot batch job application from scdf. The job runs perfectly when I click launch and run every time without using scheduler. But on using a scheduler to run the task repeatedly after few runs the job fails with below exception. I notice this problem only when I schedule the task using scheduler.

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
10-06-2020 12:00:37.633 [main] ERROR org.springframework.boot.SpringApplication.reportFailure - Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:787)
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:768)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:322)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
    at edu.capella.cufacdataloader.SeiCuFacExtractorApplication.exitApplication(SeiCuFacExtractorApplication.java:57)
    at edu.capella.cufacdataloader.SeiCuFacExtractorApplication.main(SeiCuFacExtractorApplication.java:52)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
    at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
    at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: org.springframework.batch.core.repository.JobExecutionAlreadyRunningException: A job execution for this job is already running: JobExecution: id=14665, version=1, startTime=2020-06-10 11:59:23.195, endTime=null, lastUpdated=2020-06-10 11:59:23.196, status=STARTED, exitStatus=exitCode=UNKNOWN;exitDescription=, job=[JobInstance: id=3205, version=0, Job=[CU-Job]], jobParameters=[{-spring.datasource.username=USERNAME, -spring.cloud.task.name=Alljobs, spring.batch.job.names=JOB_NAME, -spring.datasource.password=ACTUAL_PASSWORD, -spring.datasource.driverClassName=oracle.jdbc.OracleDriver, -spring.datasource.url=DATASOURCE_URL}]
    at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:116)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
    at com.sun.proxy.$Proxy100.run(Unknown Source)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:192)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:166)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:153)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:148)
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:784)

I also checked the batch_job_execution_params table and it's printing the all of the application.properties as params used by the custom SCDF server. Below is the snippet of the parameters passed. All arguments are loaded in parameters table.

One of the scheduled task arguments:

But these arguments differ when the task has been executed manually once at a time. Manually executed task arguments:

My job bean and incrementer:

    @Bean
    public Job loadJob() {
        return jobBuilderFactory
                .get(JOB_NAME)
                .incrementer(new SampleIncrementer())
                .start(step_job_details_crse()).on("COMPLETED")
                .to(step01())
                .end()
                .listener(jobExecutionListener).build();
    }

@Component
public class SampleIncrementer implements JobParametersIncrementer {
    Logger logger = LoggerFactory.getLogger(SampleIncrementer.class);
    public JobParameters getNext(JobParameters parameters) {
        logger.info("In sample incrementer");
        JobParametersBuilder param = new JobParametersBuilder();
        param.addString("JobID", String.valueOf(System.currentTimeMillis()));
        return new JobParametersBuilder().toJobParameters();
    }
}

How can I execute them repeatedly without getting the above exception? Also is there way to stop passing all arguments as job parameter?

(For some reason the properties of config-maps which should contain the application properties are loaded by deployment.yml fil , so I fallback to adding application.properties in customer built scdf projecvt.)

This exception means you are trying to run a job instance for which there is a job execution already running. However, since you are configuring a `JobParametersIncrementer` that adds a distinct JobID parameter each time, you should have a different job instance on each run. So I don't see how you could have this exception. Please share a minimal repo that reproduces the issue to be able to help you. — Mahmoud Ben Hassine, Jun 10 '20 at 12:40
Hi, I have uploaded a sample project with the same design i use at [StudentBatchProject](https://github.com/Srkanna/SCDF_CustomBuild_2.5.1/tree/master/student). Also attached the SCDF Deployement.yml and config files for reference. — Rajesh2389, Jun 10 '20 at 18:35
Thank you for sharing the sample. I took a look an unfortunately that's not a [minimal](https://stackoverflow.com/help/minimal-reproducible-example) example, there is a lot of code probably not related to the issue and I could not run it (it does not use an in-memory db, etc by looking at your deployment descriptor). But I will give you some guidelines: you need to make sure that each time you launch a job from your scheduler, that you have a new job instance (aka pass different identifying parameters). — Mahmoud Ben Hassine, Jun 16 '20 at 07:48
Hi, yes it was big one. Thought it could also because of the design issue. so shared as it is. I found the problem and it's silly mistake. In the SampleIncrementer I always return the new JobParametersBuilder instance instead of the built one. After I returned the constructed param properly that it worked fine. I got stuck with the [SCDF Job parameter problem](https://stackoverflow.com/questions/62402860/db-credentials-exposed-as-part-job-parameters-when-executing-a-task-from-scdf) for last few days. So I couldn't comment it early. — Rajesh2389, Jun 16 '20 at 08:15
ok glad you found your issue. I will try to help on the other question. — Mahmoud Ben Hassine, Jun 16 '20 at 08:18

A job instance already exists when scheduling through spring cloud dataflow server

0 Answers0