1

I referred to the example posted here. I am trying to run Multiple spring cloud stream application together. Here the output of first is given as input to other. Below is what I am trying to do. 

@Bean
    public Function<KStream<FormUUID, FormData>, KStream<UUID, Application>> process()
    {
        //do some processing here and return 
    }
// read output from above process and join it with an event stream
@Bean
    public BiConsumer<KStream<UUID, ProcessEvent>, KTable<UUID, Application>> listen()
    {

        return (eventStream,appTable )-> eventStream
                .join(appTable, (event, app) -> app).foreach((k, app) -> app.createQuote());

    }

The application.yml looks like below

spring.cloud:
 function: process;listen
 stream:
  kafka.streams:
    bindings:
      process-in-0.consumer.application-id: form-aggregator
      listen-in-0.consumer.application-id: event-processor
      listen-in-1.consumer.application-id: event-processor
    binder.configuration:
      default.key.serde: org.springframework.kafka.support.serializer.JsonSerde
      default.value.serde: org.springframework.kafka.support.serializer.JsonSerde
      spring.json.key.default.type: com.xxx.datamapper.domain.FormUUID
      spring.json.value.default.type: com.xxx.datamapper.domain.FormData
      commit.interval.ms: 1000
  bindings:
    process-in-0.destination: FORM_DATA_TOPIC
    process-out-0.destination: APPLICATION_TOPIC
    listen-in-0.destination: APPLICATION_TOPIC
    listen-in-1.destination: PROCESS_TOPIC

Above configuration throws 

java.lang.IllegalStateException: Multiple functions found, but function definition property is not set.

if I try to use below configuration 

spring.cloud.stream.function.definition: processAndListen

Then my application works but the second stream config (defined in listen Bean) doesnt gets executed.

Anupam Gupta
  • 1,591
  • 8
  • 36
  • 60

1 Answers1

4

In your property, you need to add this:

spring.cloud:
 function.definition: process;listen

This should also work - spring.cloud.stream.function.definition: process;listen.

What is processAndListen. Where does that value come from?

sobychacko
  • 5,099
  • 15
  • 26
  • Thanks the above configuration worked!! However, now I am facing another SerializationException. The problem is "spring.json.key.default.type" property is already taken and I want to configure the type for remaining types UUID, Application and ProcessEvent. Actually the mapping type is needed for each consumer and producer defined. Please point me to right direction on how I can do this. Also not sure if followup question shall be asked here or I shall create another question. – Anupam Gupta May 19 '20 at 13:38
  • Hi, I suggest that you ask another question since its unrelated. Please provide a sample application if possible or code snippets for what you are trying to accomplish. – sobychacko May 19 '20 at 14:28
  • 1
    I have created a new question for my comment https://stackoverflow.com/questions/61897937/creating-a-kafka-aggregator-and-joining-it-with-an-event – Anupam Gupta May 19 '20 at 18:14
  • For me second property worked - "spring.cloud.stream.function.definition: process;listen" – RDK Jun 10 '20 at 15:04