0

I have referred this. but, this is an old post so i'm looking for a better solution if any.

I have an input topic that contains 'userActivity' data. Now I wish to gather different analytics based on userInterest, userSubscribedGroup, userChoice, etc... produced to distinct output topics from the same Kafka-streams-application.

Could you help me achieve this... ps: This my first time using Kafka-streams so I'm unaware of any other alternatives.

edit:

It's possible that One record matches multiple criteria, in which case the same record should go into those output topics as well.

if(record1 matches criteria1) then... output to topic1;
if(record1 matches criteria2) then ... output to topic2;
and so on.
note: i'm not looking elseIf kind of solution.
Sudarshan
  • 37
  • 7

1 Answers1

1

For dynamically choosing which topic to send to at runtime based on each record's key-value pairs. Apache Kafka version 2.0 or later introduced a feature called: Dynamic routing

And this is an example of it: https://kafka-tutorials.confluent.io/dynamic-output-topic/confluent.html

Linh Vu
  • 736
  • 3
  • 7
  • okay thank you, I understand that with this approach One record can go into one of the output topics. The same record cannot go into multiple topics. Am I right? – Sudarshan Nov 29 '21 at 11:28
  • Actually, I'm looking for a way to send the same record into multiple topics if it matches certain criteria. – Sudarshan Nov 29 '21 at 11:30
  • I've edited the question for better understanding – Sudarshan Nov 29 '21 at 11:45
  • Your use-case, actually, for each `if(recordX matches criteriaN) then... output to topicN;` I will create **a KStream instance** with `.filter` is the condition filter in IF clause and `.to(topicN)`. – Linh Vu Nov 29 '21 at 12:07