0

We have a need to develop on-demand applications on the lines of FRP (something similar to AWS Lambda) on our private Cloud Foundry stack. This is in order to save cost on otherwise always running low volume applications.

The need is to trigger/start the application only when a message comes on our JMS based messaging system (e.g. IBM MQ). Being a low volume application, it should remain stopped all other times.

Based upon the detailed study and search through Spring documents, it appears to me that, a desired solution that triggers/start my microservice whenever a message appears on the source queue and shuts it down when done, will need a combination of: Spring Cloud Data Flow + Spring Cloud Stream + Spring Cloud Task as the technology stack.

Out of the various samples, below combination looks like the one needed to start PoC on local machine:

https://github.com/spring-cloud/spring-cloud-task/tree/master/spring-cloud-task-samples/tasksink https://github.com/spring-cloud/spring-cloud-stream-samples/tree/master/sink https://github.com/spring-cloud/spring-cloud-dataflow/tree/master/spring-cloud-dataflow-server-local

However am still unsure how to plug it all and achieve the results.

Some progress is made but the full clarity of architecting this solution is still unclear.

To best of my understanding it appears that we need to...

  1. run a Spring Cloud DataFlow Server
  2. run Spring Cloud DataFlow Shell or UI
  3. create Spring Cloud Stream sink application and run it (do we need to install it in Maven as well?)
  4. create Spring Cloud Task sink application and install it in Maven Repo
  5. do I need a Spring Cloud Task as well created?
  6. register modules (but what are modules for this case?)
  7. create Stream in DF Server and deploy on it
  8. anything more?

...to achieve the results.

Can someone help me in this and suggest is any building block is missed out from here? And if the answers to questions above are known.

Also are there any gaps in my understanding, and am I assuming it correctly enough that @EnableTask will make my application run only when triggered and keep it stopped otherwise?

Ashwin Gupta
  • 920
  • 1
  • 11
  • 17

1 Answers1

0

There is an important distinction you need to make when choosing the Spring Cloud Stream vs Spring Cloud Task applications.

Spring Cloud Stream -> Long lived micro service application that is event driven Spring Cloud Task -> Short lived micro service application that runs on demand

am I assuming it correctly enough that @EnableTask will make my application run only when triggered and keep it stopped otherwise

Yes, but the task application is not stopped instead it dies after completing its run.

create Spring Cloud Stream sink application and run it (do we need to install it in Maven as well?)

Not sure why you need a sink application. You would need a source application here that triggers the TaskLaunchRequest to the sink application. Your source application needs to be configured to receive the messages out of your JMS broker that cause the triggers.

create Spring Cloud Task sink application and install it in Maven Repo

This is the Spring Cloud stream sink application that launches the task you create at 5. This is one such example. This application is a long-lived one that looks for incoming TaskLaunchRequests

do I need a Spring Cloud Task as well created?

Yes, this is the actual task application that is the short-lived one that you want to run on demand.

Glorfindel
  • 21,988
  • 13
  • 81
  • 109
Ilayaperumal Gopinathan
  • 4,099
  • 1
  • 13
  • 12
  • By the way, the Spring Cloud team is working on supporting `function` as a deployment unit by featuring `Reactor` in that role prominently. Tune in for more updates on this. – Ilayaperumal Gopinathan Oct 27 '16 at 13:43
  • @Ashwin: Checkout this [sample](https://github.com/sabbyanandan/thumbnail-generator/) that includes stream, task, and dataflow in the solution. – Sabby Anandan Oct 27 '16 at 19:00
  • Thanks @IlayaperumalGopinathan for your responses. Couldn't reply sooner as have been away for this while. I understand that a task dies (rather than stopped) on its completion and that is in fact in lines of functional prog model I suppose. So that's fine. On the note of a "Stream sink application"- what else will be my source app type? Will it be a simple JMS listener app? – Ashwin Gupta Nov 02 '16 at 04:02
  • @SabbyAnandan thanks a lot for the sample. Answers quite a bit of my questions. Given I need to use a JMS source than file, can you please also given some pointers on help with desired commands? I am willing to access documentation on ```stream``` and other Dataflow Shell command in order to understand and use them for various use cases/patterns. Is there any available? – Ashwin Gupta Nov 02 '16 at 06:11
  • Yeah, the `source` application is just a producer of triggers to launch tasks based on the events received from JMS. JMS source is available as out of the box apps from spring cloud stream app starters: https://github.com/spring-cloud/spring-cloud-stream-app-starters/tree/master/jms/spring-cloud-starter-stream-source-jms. You would need to customize this Spring Cloud Stream application that generates triggers based on the messages received from JMS. – Ilayaperumal Gopinathan Nov 02 '16 at 08:42
  • Thanks @IlayaperumalGopinathan. This looks promising and goes well with the sample Sabby has shared. Will give it a try. Thanks again! – Ashwin Gupta Nov 02 '16 at 09:49