3

I have been looking at using projects built using spring-cloud-task within spring-cloud-dataflow. Having looked at the example projects and the documentation, the indication seems to be that tasks are launched manually through the dashboard or the shell. Does spring-cloud-dataflow provide any way of scheduling task definitions so that they can run for example on a cron schedule? I.e. Can you create a spring-cloud-task app which itself has no knowledge of a schedule, but deploy it to the dataflow server and configure the scheduling there?

Among the posts and blogs I have looked at I noticed the following:

https://spring.io/blog/2016/01/27/introducing-spring-cloud-task

Some of the Q&A afterwards hints at this being a possibility, with the reference to triggers, but I think this was discussed before it was released.

Any advice would be greatly appreciated, many thanks.

Nigel R
  • 71
  • 1
  • 4

2 Answers2

2

There are few ways you could launch Tasks in Spring Cloud Data Flow. Following are the available options today.

  • Launch it using TriggerTask; with this you could either choose to launch it with fixedDelay or via a cron expression - example here.
  • Launch it via an event in streaming pipeline. Imagine a use-case where you would want to create a "thumbnail" as and when there's a new image (event) in s3-bucket or in a file-system directory; the "thumbnail" operation could be a task in this case - example here.

Lastly, in the upcoming releases, we will port over "scheduler" functionality from Spring XD to Spring Cloud Data Flow.

Sabby Anandan
  • 5,636
  • 2
  • 12
  • 21
  • Many thanks both for your help - the TriggerTask does seem ideal for this. I have a further question: I cannot seem to get the cron parameter to work correctly. I have tried as a stream definition e.g. triggertask --uri="file://C:/Software/timestamp-task-1.0.1.RELEASE.jar" --cron="0 0 12 * * ?" | task-sink but it ignores this and fires once per second. Using --fixed-delay and --time-unit instead work fine. Is the above syntactically correct? Have searched docs and other posts but can't see an example which explicitly uses --cron. – Nigel R Aug 18 '16 at 09:54
  • There was a recent bug-fix to `triggertask` source application - see [#201](https://github.com/spring-cloud/spring-cloud-stream-app-starters/pull/201). Until we resolve the CI failures, you could build this application locally - follow [this procedure](https://github.com/spring-cloud/spring-cloud-stream-app-starters/blob/master/README.adoc). You can then force-register 1.0.3.BUILD-SNAPSHOT release of `triggertask` in SCDF. One other thing, you need to use property-prefix for `--trigger.cron` as described in the docs. – Sabby Anandan Aug 18 '16 at 14:02
  • CI build is now complete and 1.0.3.BUILD-SNAPSHOT variant of static file is generated. Use the static file (_eg. for kafka: dataflow:>app import --uri http://repo.spring.io/libs-snapshot/org/springframework/cloud/stream/app/spring-cloud-stream-app-descriptor/1.0.3.BUILD-SNAPSHOT/spring-cloud-stream-app-descriptor-1.0.3.BUILD-SNAPSHOT.stream-apps-kafka-maven --force_) to bulk import new version of the apps, which includes the bug-fix to `triggertask` application. – Sabby Anandan Aug 18 '16 at 15:34
  • 6
    Any word on the status of the Spring XD scheduler support? – Casey May 02 '17 at 20:47
  • I am getting `Scheduling is not implemented for local platform.` on trying to schedule the job – Tomas Lukac Sep 27 '21 at 08:27
2

Yes, Spring Cloud Data Flow does provide a scheduling option. To enable it, you need to add below arguments while starting the server:

--spring.cloud.dataflow.features.schedules-enabled=true
Adrian Mole
  • 49,934
  • 160
  • 51
  • 83
Hemant
  • 21
  • 3