2

I am fairly new to Kafka and just started with Python, so pardon my wrongful terms, I'll try to explain my gist.

I have a Kafka queue, where my topics are being generated pragmatically. Now these topics will hold a list of orders and will be picked up by consumers to process. My requirement is that, if there are multiple orders within a topic, the subsequent order should only be processed when the previous order is successful, if the order fails I want it to be retried. Also orders in multiple topics can be processed manually. So while reading through the docs, I fount the concept of committing the offsets, I am not sure if this is the thing that will help me, I am not able to piece it all together. Can someone help me with this?

Paras
  • 3,191
  • 6
  • 41
  • 77

1 Answers1

0

This logic is something that you have to solve outside kafka in your consumer. Kafka provides some scenarios: At-Most-Once, At-Least-Once, Exactly-Once

https://dzone.com/articles/kafka-clients-at-most-once-at-least-once-exactly-o

But it seems you have a synchronous order processing system, and you will lose some of the beneffits of using Kafka. You could try At-Most-Once and if it fails you push it back in the topic.