3

I’m setting up a new Spring Batch Jobs and want to deploy it using SCDF. However, I have found that SCDF does not support scheduler feature in local framework.

I have 3 questions to ask you:

  1. Can someone explain how scheduler of SCDF work?

  2. Are there any ways to schedule 1 job using SCDF?

  3. Can I use my local server as a Cloud Foundry? and how?

halfer
  • 19,824
  • 17
  • 99
  • 186
Zach Pham
  • 280
  • 4
  • 18
  • Welcome to SO! When you place a question try to add a minimum content: input sample, expected output sample, what did you try, research and where are you stacked. What did you try? What about your research? – David García Bodego Oct 21 '19 at 04:28
  • Please read [Under what circumstances may I add “urgent” or other similar phrases to my question, in order to obtain faster answers?](//meta.stackoverflow.com/q/326569) - the summary is that this is not an ideal way to address volunteers, and is probably counterproductive to obtaining answers. Please refrain from adding this to your questions. – halfer Oct 22 '19 at 20:13
  • @user158: re your edit here, please do not add bold to whole lines or paragraphs - it does not make things easier to read. Similarly, please do not add code formatting to things that are not actually code (or console I/O). – halfer Oct 22 '19 at 20:15

1 Answers1

2

Yes, Spring Cloud Data Flow does not support scheduling on local platform. Please note that the local SCDF server is for development purposes only and by design, the scheduling support is intended to be relying on the platform. Hence, SCDF scheduling feature is supported on Cloud Foundry and Kubernetes using the CF and K8s schedulers.

1) Can s/o explain how scheduler of SCDF work?

sure, Similar to how the deployer is used for launching task/deploying the stream, there is an SPI for scheduling the tasks under spring-cloud-deployer project. The underlying scheduler implementations can implement this. Currently, we have CF and K8s scheduler implementations in spring-cloud-deployer-cloudfoundry and spring-cloud-deployer-kubernetes.

As a user, you can configure a scheduler for a task (batch) application (via SCDF Dashboard, shell etc.,). You can specify a cron expression to schedule the task. Once configured, the SCDF delegates the schedule request to the platform scheduler using the above-mentioned scheduler implementations. Once scheduled, it is the platform (PCF scheduler on CF, K8s scheduler on K8s) that takes care of the task using the schedule.

2) Are there any ways to schedule 1 job using SCDF?

Yes, based on the answer from 1

3) Can I use my local server as a cloud Foundry? and How?

To run SCDF on local pointing to the CF instance, you can set the necessary CF deployer properties and start the SCDF server instance. It is similar to how you configure multi platforms in SCDF server. You can find more documentation on this here.

Ilayaperumal Gopinathan
  • 4,099
  • 1
  • 13
  • 12
  • Thanks for answering. so can I have scheduler feature while I use my local server as a cloud Foundry? – Zach Pham Oct 21 '19 at 06:56
  • Yes, as long as you have the scheduling features enabled and scheduler URL pointing to the correct scheduler instance. – Ilayaperumal Gopinathan Oct 21 '19 at 10:30
  • I have used spring.cloud.dataflow.features.schedules-enatbled: true in application.properties. Is that scheduling features which you have mentioned? However, I do not know how to set "scheduler URL pointing to the correct scheduler instance". Do you have any document or user manual about it? please give me some examples? Thanks so much for your help. – Zach Pham Oct 22 '19 at 01:31
  • You need to set the `SPRING_CLOUD_SCHEDULER_CLOUDFOUNDRY_SCHEDULER_URL` property to the value: https://scheduler. – Ilayaperumal Gopinathan Oct 22 '19 at 07:03
  • In conclusion, I have to install cloud foundry on local server (mentioned in this topic: https://stackoverflow.com/questions/36596743/how-to-install-cloudfoundry-on-local-server). After that, I can point to the CF instance on that "local cloud" to use scheduler feature of SCDF. Is it right? – Zach Pham Oct 22 '19 at 08:32
  • Though it is possible to get this up this way. One more thing, when running local, you also need to set the `spring.profiles.active=cloud` and set `VCAP_APPLICATION` to some valid JSON. Again, I want to repeat that this is not the recommended way of running the SCDF server instance. You are expected to run the SCDF server on CF. – Ilayaperumal Gopinathan Oct 23 '19 at 11:59
  • how about the Task Scheduler Launcher? does it use to schedule task on local? – Zach Pham Oct 29 '19 at 12:01
  • Which SCDF release version are you on? The Task Scheduler Launcher is an ongoing feature and is not part of any of the releases yet (it's currently on master and we don't have the documentation updated yet). So, please try any of the released versions of SCDF – Ilayaperumal Gopinathan Oct 29 '19 at 16:20