0

I'm developing a custom starter app using the code here, because I wanted to add TcpSSLContextSupport to it. I've managed to do that, and it worked fine on my local SCDF server, but now I want to deploy it on SCDF for Kubernetes over Google Cloud. I've created an image for it and it does work, but the behavior is a little unexpected.

I created a stream like: stream create stream-name --definition "app-name | log".

First off, it created a topic as per convention stream-name.app-name, and I was expecting it to show me the messages there, but it actually writes into the "output" topic, which is strange. I haven't provided any additional configuration. I tried it with SCDF latest version as well as 1.2.0.RELEASE with the same results.

ystark
  • 565
  • 9
  • 18

1 Answers1

0

Since the topic stream-name.app-name is created, the SCDF correctly sets the outbound destination name for your source app. Note that SCDF sets only the property spring.cloud.stream.bindings.output.destination for the apps. Here, the outbound channel is assumed to be output.

I guess you have a different binding target name for the outbound message channel in your custom application. Since SCDF doesn't set the destination for this app, the app defaults to output as the destination.

Check your outbound target name in your custom app. You need to set the property spring.cloud.stream.bindings.<customOutboundTargetName>.destination as a deployment property for the app.

For instance,

dataflow:>stream deploy stream-name --properties "app.source-app-name.spring.cloud.stream.bindings.customOutboundTargetName.destination=yourTopic,app.sink-app-name.spring.cloud.stream.bindings.customInboudTargetName.destination=yourTopic"

Note: customOutboundTargetName and customInboundTargetName are set as output and input by default via SCDF.

Maroun
  • 94,125
  • 30
  • 188
  • 241
Ilayaperumal Gopinathan
  • 4,099
  • 1
  • 13
  • 12
  • I know about the configuration that I can add, but I'm trying to understand why Spring is doing this specifically with my custom app, but not when I use the `tcp | log` option, especially when I haven't changed anything that I can think of. And I also don't know what you mean when you said that Spring correctly sets the outbound channel when `stream-name.app-name` is created, because it clearly sets the outbound channel to `output` despite that? – ystark May 26 '17 at 05:54
  • All the out of the box apps come up with the outbound and inbound channel names as `output` and `input`. That's the reason SCDF sets the destination name using that channel name. If you use a different channel name in your custom app, then you need to configure as above. – Ilayaperumal Gopinathan May 26 '17 at 09:39
  • 1
    Also, it is Spring Cloud Stream that sets the outbound destination name (topic) as `output` and it happens here: https://github.com/spring-cloud/spring-cloud-stream/blob/master/spring-cloud-stream/src/main/java/org/springframework/cloud/stream/config/BindingServiceProperties.java#L207 – Ilayaperumal Gopinathan May 26 '17 at 09:41
  • That makes sense. But why does it create the `stream-name.app-name` topic at all when it's going to write into the `output` topic? – ystark May 26 '17 at 11:29
  • I think that's because SCDF also assigns `spring.cloud.stream.bindings.output.destination=stream-name.app-name`. I think SCDF can improvise this. Created https://github.com/spring-cloud/spring-cloud-dataflow/issues/1519 to track it. Thanks! – Ilayaperumal Gopinathan May 26 '17 at 12:08
  • So if `output` is the default, then why doesn't the log-sink consume from it? The log-sink consumes from the topic `stream-name.app-name`, which has nothing, as all the data is going into `output`. – ystark May 29 '17 at 04:56
  • Also, I should probably reiterate that with SCDF local and my app registered as a maven artifact, it works as expected by writing to the topic `stream-name.app-name`. But when I create a container and deploy it in SCDF for Kubernetes, the messages are written in `output`. – ystark May 29 '17 at 05:21
  • `output` is the default `destination` (topic) name where as SCDF sets the topic name as `stream-name.app-name`. Since you have a custom named channel which isn't set with a specific destination name, the default `output` topic is created. – Ilayaperumal Gopinathan May 29 '17 at 05:22