1

Is there a way in Kafka for a client to subscribe to a topic to get a snapshot/state of the world "KTable" when first connect and also subscribe to the subsequent updates?

Say we have a Topic of records like below 

|Key   | Value |
|:-----|------:|
| a    |  vala |
| b    |  valb |
| a    |  valc | 
| d    |  vald |
| d    |  vale |

And when any client (existing or new) connect/reconnect, I'd like to one-off give them

|Key   | Value |
|:-----|------:|
| b    |  valb |
| a    |  valc | 
| d    |  vale |

And subscribe them to further updates as a stream.

user2001850
  • 477
  • 4
  • 18
  • There is no build-in feature for this atm. There is some ongoing work in ksqlDB that may support this use case in the future though: https://ksqldb.io – Matthias J. Sax Apr 18 '20 at 21:56
  • @MatthiasJ.Sax thanks. Do you have any recommendations of any pattern that can be used for this kind of scenario? My priority is transparent experience to client (client doesn't need to switch channel from state of the world to stream). – user2001850 Apr 19 '20 at 15:15
  • Given the current API of Kafka Streams and ksqlDB, it seems hard to built it manually using either one of both. The best best might be, to fall back using a plain `KafkaConsumer` that gives you more flexibility that you may need to build it from scratch. Don't have any more detailed advice. Sorry. – Matthias J. Sax Apr 19 '20 at 18:07

0 Answers0