0

I'am using spring-cloud-data-flow-server-yarn and successfully deployed my spring-cloud-stream apps in my yarn-cluster.

Now I'm facing a problem:

When I deploy my scs apps, the property value of spring.cloud.stream.bindings.output.destination will be override to streamName.groupName, which I think is the default value for the channel name.

How can I let the properties of the channel name inside the scs-app take effective? I mean not using the deploy properties, but just let the properties inside the scs-app take effective.

HDJEMAI
  • 9,436
  • 46
  • 67
  • 93
zjunothing
  • 61
  • 3

1 Answers1

0

Spring Cloud Data Flow assumes the inbound/outbound target (ex: message channel) names to be input/output based on the OOTB apps. If you have a custom application with the target channel names different than input/output then, you would need to set the destination names explicitly via stream definition/deployment properties.

There is a Github issue on this and you can track it from there.

EDIT:

Some more general points to be noted:

SCDF needs to know the group name (here the Stream name) in the context of a stream being deployed. Whereas, as a standalone Spring Cloud Stream application it has its own way of setting consumer group name etc., in the context of an application level.

SCDF is built as an orchestration model to run Streams/Tasks using Spring Cloud Stream/Task applications. The binding destination properties are one of those that are required to be overridden by SCDF to adhere to this Stream model. But, you can still override this setting of destination via deployment properties

Ilayaperumal Gopinathan
  • 4,099
  • 1
  • 13
  • 12
  • Thanks for your quick reply Ilayaperumal. Let me give an example: I have a custom source-app named my-source-app, and inside this app, i have a property `spring.cloud.stream.bindings.output.destination=time-topic` , and when i register it as an app in scdf(named it my-source-app) and using it to compose a stream(named it my-stream), and deploy it into my yarn-cluster(without any explicit deployment properties), then the channel destination will be `my-stream.my-source-app` but not `time-topic`. I want it to be `time-topic` which means the properties inside my app can take effective – zjunothing Jun 01 '17 at 02:17
  • Properties inside the app get the least precedence and hence the override happens. As you colluded in the question, you need to set `app.myapp.spring.cloud.stream.bindings.output.destination=time-topic` as part of deployment properties when you deploy the stream. – Ilayaperumal Gopinathan Jun 01 '17 at 03:18
  • You said the properties inside the app have the least precedence. But if i didn't set any deployment properties, ( no higher precedence properties there) , the properties inside the app should take effective right ? – zjunothing Jun 01 '17 at 04:02
  • No, since SCDF sets the binding destination properties explicitly via deployment request it has the higher precedence over the one inside the app. Hence, we need to override this via deployment properties – Ilayaperumal Gopinathan Jun 01 '17 at 04:19
  • So it just means the binding destination properties inside your scs app is of no use when you using SCDF to deploy the scs app. It's a bit of counter-intuitive. – zjunothing Jun 01 '17 at 05:48
  • I think the intuitive way should be: if i provide the deployment property of binding destination, then it should override the binding destination inside my SCS app, if i didn't provide the deployment property of binding destination, then the property inside my SCS app should take effective. – zjunothing Jun 01 '17 at 05:54
  • 1
    SCDF needs to know the group name (here the Stream name) in the context of a stream being deployed. Whereas, as a standalone Spring Cloud Stream application it has its own way of setting consumer group name etc., in the context of an `application` level. SCDF is built as an orchestration model to run Streams/Tasks using Spring Cloud Stream/Task applications. The binding destination properties are one of those that are required to be overridden by SCDF to adhere to this Stream model. But, as we discussed above you can still override via deployment properties. – Ilayaperumal Gopinathan Jun 01 '17 at 06:15
  • Thanks Iilayaperumal, that make sense. – zjunothing Jun 01 '17 at 06:36