0

I'm new to Kafka and Kafka Streams. While I have gone through the concepts of Kafka and Kafka Streams and feel confident conceptually, there is one thing that's confusing me in and out. Yes, it's the decision between using Ktable vs state store.

I'm building a stateful application where I need to store the state of my application over the time. How do I decide if I should use a Ktable/Global Ktable vs a state store? Are these same or dependent on each other? To be honest, I am still not clear on the difference between the both and hence unable to make a confident decision on what shall I use for my application.

Usecase: I have a workflowId and the status pertaining to that workflowId. The status of the workflow would be determined by the underlying downstream workflow engine. I need to store/update the status of the workflowId as received from the workflow engine and persist this information as the operations on the workflowId would depend upon the status of the workflowId.

eureka19
  • 2,731
  • 5
  • 26
  • 34
  • State store is dependency of both. Regarding which to use, we need more information about your use case – OneCricketeer Aug 21 '23 at 12:27
  • @OneCricketeer Provided the use case in the post. – eureka19 Aug 21 '23 at 13:10
  • So what I understood from your comment is that I cannot use a state store as that is the implementation detail of ktable/global ktable. I need to use the ktable/global ktable and it will internally use the state store which would be based on the change-log topic and the RocksDB by default. Is this understanding correct? – eureka19 Aug 21 '23 at 13:12
  • You can "use a StateStore", yes, but it'll require using the [Processor API](https://kafka.apache.org/35/documentation/streams/developer-guide/processor-api.html#state-stores) of Kafka Streams, while the DSL will only give you access via a kind of KTable. But yes, that's the gist. And I have a friend that is also building a workflow engine, as a matter of fact - https://github.com/littlehorse-enterprises/littlehorse/tree/master/server – OneCricketeer Aug 22 '23 at 18:11
  • @OneCricketeer Thanks for your response. I still have my initial question open - what is the difference between a state store and a KTable ? – eureka19 Aug 22 '23 at 19:11
  • 1
    Just the class name ... ? As mentioned, a KTable (global or not) is built on top of a StateStore. However, KTables generally follow some windowing/aggregation pattern, and you can do lower-level operations with more customized logic with a direct store (and the Processor API overall) – OneCricketeer Aug 22 '23 at 23:16

0 Answers0