0

I want to split a topic into serveral topics, based on specific fields in the data. Is there a possibility to do such operations in the streaming itself?

Or should I write a typically Consumer and Producer?

Example:

Input Topic: Data with attributes A (values AAA, BBB, CCC), B, C, D and E

Output Topics: 3 Topics with Name AAA, BBB and CCC with the same data as in Input Topic

Kind Regards, Lukas

Lukas
  • 31
  • 1
  • 3
  • Possible duplicate of [Dynamically connecting a Kafka input stream to multiple output streams](https://stackoverflow.com/questions/41796207/dynamically-connecting-a-kafka-input-stream-to-multiple-output-streams) – Matthias J. Sax Sep 25 '17 at 20:05

1 Answers1

1

You can use branch() method in your stream to split into several streams. Than each stream can write in own topic.

hleb.albau
  • 137
  • 10
  • I want to split into several streams dynamically, so it’s not possible to use the branch method, because I should know the split criteria before. I have to split the incoming stream into hundreds of other streams. I think I can only use the consumer and producer to do this. – Lukas Sep 25 '17 at 05:34
  • Hi Lukas, as far as i know streams, you could archive such behavior only using own consumers-producers pair. Also, have a look at low-lvl api of streams, may be in sink processor you can define needed functionality. – hleb.albau Sep 25 '17 at 20:04