1

We materialize the KTable into a Internal-State-Store.

a.) How and where can I specify that, this Internal-State-Store should be Persistent and be automatically backed-up to another kafka topic ?

b.) How can we specify that, this Internal-State-Store should be global one, so that any of my stream-task should be able to refer to that ?

c.) Is there a frequency, upon which the incoming messaageRecords are being written to the Internal-State-Store ? Can it happen that, a particular MessageRecord gets processed by Stream-processor, being stored in KTable, and then my stream-processor dies and it could not make entry to Internal-State-Store !!

Below snippet we use right now :-

KTable<String, String> KT0 = streamsBuilder.table(AppConfigs.topicName, Materialized.as(AppConfigs.stateStoreName)));

Any responses shall be highly appreciated !!

Aditya Goel
  • 201
  • 1
  • 15

1 Answers1

0

a) If you have a custom implementation of a state store, you can pass it via Materialized.as(KeyValueStoreSupplier).

b) For global-store use-case, you can use builder.globalKTable().

c) Writes happen on a per record basis, but could cached in-memory. Before input topic offsets are committed, the state store will be flushed though, and thus you will never miss any data. By default, KafkaStreams provides at-least-once processing semantics.

Matthias J. Sax
  • 59,682
  • 7
  • 117
  • 137