2

I've encountered an issue on Spring Cloud Dataflow when running multiple composed tasks at once.

Hikari DataSource takes 10 connections from the connection pool by default. When running for example 10 composed tasks at once, this means 100 connections + connections required for every task on each composed task.

I tried running the Composed Task Runner locally with spring.datasource.hikari.maximum-pool-size=1 and it worked.

Is there any way how to set this property to every Composed Task Runner by default ? I did not find any documentation related to modifying things like this for composed tasks.

Tomas Lukac
  • 1,923
  • 2
  • 19
  • 37
  • `Is there any way how to set this property to **every** Composed Task Runner by default ?`: Does this mean you are running multiple Composed Task Runners (one for each composed task)? – Mahmoud Ben Hassine May 05 '22 at 04:45
  • There is one Composed Task Runner. What I meant was "every composed task execution". I can set the `spring.datasource.hikari.maximum-pool-size=1` to the SCDF server, which will be picked up by the Composed Task Runner as well, but I would like to keep the default 10 in SCDF server. – Tomas Lukac May 05 '22 at 05:05
  • Hello @TomasLukac, In short no. Please create an issue on Spring Cloud Data Flow. You can set the property at the time you create the composed task definition. Docs on doing this can be found here: https://dataflow.spring.io/docs/feature-guides/batch/composed-task/#passing-properties-to-composed-task-runner – Glenn Renfro May 09 '22 at 11:55

0 Answers0