0

I am trying to join two streams in Kafka like following code.

@Bean
public KTable<String, String> stream(StreamsBuilder builder) {
            KeyValueBytesStoreSupplier store =
            Stores.persistentKeyValueStore("words");
    JsonSerde<String> wordSerde = new JsonSerde<>(String.class);
    KStream<String, String> stream = builder
            .stream("topic-a", Consumed.with(Serdes.String(), wordSerde));

    return stream.join(
                    builder.stream("topic-b"),
                    (w1, w2) -> w1+w2,
                    JoinWindows.of(Duration.ofSeconds(10)),
                    StreamJoined.with(Serdes.String(), wordSerde, wordSerde))
            .toTable(Materialized.<String, String>as(store)
            .withKeySerde(Serdes.String())
            .withValueSerde(wordSerde));

}

but when I'm trying to read from KeyValueBytesStoreSupplier it is just only reading from topic-a.

updated: I published some messages to both topics. for example in topic-a I sent Hi message and same message in topic-b what thing that I expect is merging two words together. HiHi

@PostMapping("publish/{word}")
fun publishEvent2(@PathVariable(value = "word") event: String){
   kafkaTemplate2.send("topic-a", event,event)
}

Did I miss somethings?

Emil
  • 423
  • 1
  • 12
  • 34
  • 2
    Can you add to the question some example records you are sending to both topics? – Paweł Szymczyk Nov 17 '22 at 19:46
  • @PawełSzymczyk I did, does make sense? let me know if you need more information. – Emil Nov 17 '22 at 20:22
  • 1
    Please specify messages in format: key: xyz, value: qwe. Join is made by messages key, so if you not specify the same key, join will not be executed. – Paweł Szymczyk Nov 17 '22 at 20:31
  • @PawełSzymczyk I'm using spring boot. so for sending message I use kafkaTemplate. see updated. same for topic-b as well. – Emil Nov 17 '22 at 20:58
  • unrelated: your data is just a string, not JSON, so you should use Serdes.String rather than wordSerde – OneCricketeer Nov 18 '22 at 00:14
  • Are you sending the data within the same 10 seconds? If not, make your window larger. If you dont want a time window, then make topic-a into a KTable, then join topic-b against that... – OneCricketeer Nov 18 '22 at 00:17
  • 1
    I've just copied the code and everything works as expected. I've changed two things: window time to ten minutes, and value serde to Serdes.String() – Paweł Szymczyk Nov 18 '22 at 13:42
  • @PawełSzymczyk Yes, last night tried again and it was ok. :( I'm sorry for silly question. – Emil Nov 19 '22 at 00:01

0 Answers0