I am trying to pass global properties into Composed tasks. When I specify each and every job, it works, but there is configuration duplicity. I tried multiple approaches, but the properties are not set correctly.
My project (jar) contains multiple Spring Batch jobs. Which one is executed depends on the java argument. The CSDF task looks like this:
download: my-task 'COMPLETED'->status_ok: my-task 'FAILED'->status_error: my-task
There is also an option in SCDF UI for adding these global properties in the Global column:
which translates to app.*.property=value.
However, this does not work and properties are not sent over in the environment variables in each task.
Since I am using a database for my project, I need to specify datasource for the SCDF and also for JPA. Hence I am overriding DefaultTaskConfigurer. Configuration that works looks like this:
app.my-task.download.spring.batchdatasource.driver-class-name=org.mariadb.jdbc.Driver
app.my-task.download.spring.batchdatasource.password=rootpw
app.my-task.download.spring.batchdatasource.url=jdbc:mysql://mysql:3306/dataflow
app.my-task.download.spring.batchdatasource.username=root
app.my-task.download.spring.datasource.driver-class-name=org.postgresql.Driver
app.my-task.download.spring.datasource.password=pw
app.my-task.download.spring.datasource.url=jdbc:postgresql://postgres:5432/csv
app.my-task.download.spring.datasource.username=root
app.my-task.status_ok.spring.batchdatasource.driver-class-name=org.mariadb.jdbc.Driver
app.my-task.status_ok.spring.batchdatasource.password=rootpw
app.my-task.status_ok.spring.batchdatasource.url=jdbc:mysql://mysql:3306/dataflow
app.my-task.status_ok.spring.batchdatasource.username=root
app.my-task.status_ok.spring.datasource.driver-class-name=org.postgresql.Driver
app.my-task.status_ok.spring.datasource.password=pw
app.my-task.status_ok.spring.datasource.url=jdbc:postgresql://postgres:5432/csv
app.my-task.status_ok.spring.datasource.username=root
app.my-task.status_error.spring.batchdatasource.driver-class-name=org.mariadb.jdbc.Driver
app.my-task.status_error.spring.batchdatasource.password=rootpw
app.my-task.status_error.spring.batchdatasource.url=jdbc:mysql://mysql:3306/dataflow
app.my-task.status_error.spring.batchdatasource.username=root
app.my-task.status_error.spring.datasource.driver-class-name=org.postgresql.Driver
app.my-task.status_error.spring.datasource.password=pw
app.my-task.status_error.spring.datasource.url=jdbc:postgresql://postgres:5432/csv
app.my-task.status_error.spring.datasource.username=root
batchdatasource.* properties are for the SCDF and datasource.* is for my personal use. This works like a charm, both datasources are initialized.
Also, run arguments look like this:
app.download.0=--jobName=download
app.status_error.0=--status=Error
app.status_error.1=--jobName=status
app.status_ok.0=--status=Ok
app.status_ok.1=--jobName=status
But as you can see, there is a lot of duplicity. I tried to do it this way
app.*.spring.batchdatasource.driver-class-name=org.mariadb.jdbc.Driver
app.*.spring.batchdatasource.password=rootpw
app.*.spring.batchdatasource.url=jdbc:mysql://mysql:3306/dataflow
app.*.spring.batchdatasource.username=root
app.*.spring.datasource.driver-class-name=org.postgresql.Driver
app.*.spring.datasource.password=pw
app.*.spring.datasource.url=jdbc:postgresql://postgres:5432/csv
app.*.spring.datasource.username=root
and this way
app.my-task.*.spring.batchdatasource.driver-class-name=org.mariadb.jdbc.Driver
app.my-task.*.spring.batchdatasource.password=rootpw
app.my-task.*.spring.batchdatasource.url=jdbc:mysql://mysql:3306/dataflow
app.my-task.*.spring.batchdatasource.username=root
app.my-task.*.spring.datasource.driver-class-name=org.postgresql.Driver
app.my-task.*.spring.datasource.password=pw
app.my-task.*.spring.datasource.url=jdbc:postgresql://postgres:5432/csv
app.my-task.*.spring.datasource.username=root
but none of this works. What am I doing wrong ?
As you can see, properties were not set for this task execution.
