Normally we run the Jar of spring cloud data flow in one of the machine, but what if over the period we create many flows on the machine and the server gets overloaded and becomes a single point of failure, Do we have some thing where we can run the spring cloud data flow server jar on another machine and shift the flows on to that so that we can avoid any such failures and make our complete system more resilient and robust. or does the expansion happen automatically when we deploy our complete system on PCF/or cloud foundry.
Asked
Active
Viewed 1,117 times
1 Answers
2
SCDF is a simple Boot application. It doesn't retain any state about the stream/task applications itself, but it does keep track of the DSL definitions in the database.
It is common to provision multiple instances of SCDF-server and a load balancer in front for resiliency.
In PCF specifically, if you scale the SCDF-server to >1, PCF will automatically load-balance the incoming traffic (from SCDF Shell/GUI). It is also important to note that PCF will automatically restart the application instance, if it goes down for any reason. You will be set up for multiple levels of resiliency this way.
Sabby Anandan
- 5,636
- 2
- 12
- 21
-
Can you please provide the command or steps to increase number of Instance for any application in SCDF ? as i have 3 app registered. source | processor | sink. I want to allocate 2 instance to processor app. – Pratik Joshi Oct 23 '17 at 08:17
-
You'd simply issue `cf scale -i 2
` from the CF CLI. Alternatively, if you know the scaling requirements beforehand, you can deploy a stream directly with "n" no. of apps using the deployment properties _(e.g., `deployer. – Sabby Anandan Oct 23 '17 at 12:31.count=2)_ - see docs [here](https://docs.spring.io/spring-cloud-dataflow/docs/1.2.3.RELEASE/reference/htmlsingle/#_deployment_properties). -
Got it. We can add instance using below command in shell. E.G : stream deploy stream_name --properties app.app_name.count=2. This work for me. – Pratik Joshi Oct 24 '17 at 03:54