0

Is there any API or attributes that can be used or compared to determine if all messages in one topic partition are consumed? We are working on a test that will use another consumer in the same consumer group to check if the topic partition still has any message or not. One of our app services is also using Kafka to process internal events. So is there any way to sync the progress of message consumption?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Jerry
  • 1
  • 3
  • 1
    In Python? I don't think so since the consumer loop is an infinite iterator. In the Java API, you'd just check that the poll method returned zero records.... Other option is to install something like Burrow, then use its REST API to check the consumer group lag – OneCricketeer Nov 19 '21 at 19:23

1 Answers1

0

Yes you can use the admin API.

From the admin API you can get the topic offsets for each partition, and a given consumer group offsets. If all messages read, the subtraction of the later from the first will evaluate to 0 for all partitions.

Noam Levy
  • 179
  • 7