-1

i was able to fetch data form beginning to end of specific Kafka topic by using consume param(earliest). responses: offset 180 , msg abc.... . . offset 12000 , msg klp...

i want to count the number of counts from specific offet to latest offset to know how many new message being inserted. is kafka consumer has such parameter to use or i need to get groovy codes for this?

the scenario will be: start consume kafka topic, the offset might be 180-12000 post 100 msgs to kafka topic. the offset now is 180-12099, for some reason 1 msg failed to post. i need a codes to make sure total count is 100, if 99 then failed.

user2201789
  • 1,083
  • 2
  • 20
  • 45

1 Answers1

1

Instead of checking that the correct number of offsets are present, you should instead examine your producer's return codes for the messages you sent. Most kafka producers return the offset for the message when it has been acknowledged by the kafka brokers.

Dan Davies Brackett
  • 9,811
  • 2
  • 32
  • 54