0

I have a simple spring batch remote partitioning project running with spring cloud task. Im using spring cloud dataflow at run the spring batch. Spring boot version - 2.7.8 Spring cloud task - 2.4.5 spring cloud data flow - 2.10.1-SNAPSHOT

I'm using docker compose for spring cloud dataflow locally, and im using spring cloud local deployer as well.

I have spring batch to move data from one table to another within same database. I'm partitioning the data of 100 records into 4 partitions. I'm initially made one of the partition to fail.

After processsing. batch_job_execution job is failed and batch_step_execution one of the partition is failed. But the same is not reflected in scdf dashboard. In scdf dashboard, task and task execution is completed but job execution is failed status

Two questions:-

  1. How do i makesure the scdf reflect right batch status in dashboard?
  2. How do i restart the failed job execution?

For First question, i tried with "spring.cloud.task.batch.fail-on-job-failure=true" property in application.properties, but i get "Job must not be null nor empty" from TaskJobLauncherApplicationRunnerFactoryBean.java

For second question, i tried relauch the task using below REST API ->

curl 'http://localhost:9393/jobs/executions/1' -i -X PUT
-H 'Accept: application/json'
-d 'restart=true'

But it restarted all the partitiones

Sanjay
  • 11
  • 3
  • Can you provide a stacktrace? Also can you provide a sample project on github that reproduces the error.? – Glenn Renfro Feb 13 '23 at 14:34
  • @GlennRenfro Thank you for the replay. I was able to figure out the issue but forgot to update the StackOverflow. 1. To reflect the spring batch failure on to spring cloud task in SCDF. I used the "spring.cloud.task.batch.fail-on-job-failure=true" but forgot to set the "spring.batch.job.enabled=false" property on my worker deployer. Once set the SCDF was able to reflect the corresponding spring batch failure. 2. Once done I was able to restart the failed Spring cloud task via the dashboard and the processor was able to pick only the failed partition. – Sanjay Feb 14 '23 at 15:07

1 Answers1

0

@sanjay I'm glad you found a solution to your issue by setting spring.cloud.task.batch.fail-on-job-failure=true.
You can also look at the sample provided in the Spring Cloud Task here: https://github.com/spring-cloud/spring-cloud-task/blob/main/spring-cloud-task-samples/partitioned-batch-job/src/main/java/io/spring/JobConfiguration.java#LL100-L101C67

Glenn Renfro
  • 266
  • 1
  • 3