1

Im struggling to find any documentation on where i can use Spring Cloud Streams that takes a Kafka topic and puts it into a KTable. Having looked for documentation for example on here https://cloud.spring.io/spring-cloud-static/spring-cloud-stream-binder-kafka/3.0.0.RC1/reference/html/spring-cloud-stream-binder-kafka.html#_materializing_ktable_as_a_state_store nothing is very concrete on the way to do this in Spring boot using annotations. I was hoping i could just create a simple KTable using a KStream where in my application.properties i have this: spring.cloud.stream.bindings.process-in-0.destination: my-topic

Then in my Configuration I was hoping i could do something like this

@Bean
public Consumer<KStream<String, String>> process() {
    return input -> input.toTable(Materialized.as("my-store"))
}

Please advise what im missing?

Matt
  • 2,713
  • 5
  • 27
  • 35

1 Answers1

1

If all you want to do is to consume data from Kafka topic as KTable, then you can do this as below.

@Bean
public Consumer<KTable<String, String>> process() {
    return input -> {
        
    };
}

If you want to materialize the table into a named store, then you can add this to the configuration.

spring.cloud.stream.kafka.streams.bindings.process_in_0.consumer.materializedAs: my-store

You could also do what you had in the question, i.e. receive it as a KStream and then convert to KTable. However, if that is all you need to do, you might rather receive it as KTable in the first place as I suggest here.

sobychacko
  • 5,099
  • 15
  • 26
  • how does Spring know what topic to consume from in this example? I wanted to query this data to get a record by the key – Matt Apr 22 '21 at 20:46
  • You can provide the topic as you would normally in any other Spring Cloud Stream applications. In this particular example it's going to be `spring.cloud.stream.bindings.process-in-0.destination=`. – sobychacko Apr 22 '21 at 21:55
  • i also have a requirement where i need to read from this ktable in a REST style controller, how would this then feed into that? Im aware of the InteractiveQueryService so i'd suspect id need to inject that but then how does this query service become aware of this KTable? – Matt Apr 23 '21 at 06:49