0

I've researched the Kafka monitoring systems and found out that there are two types of them. First is consuming __consumer_offsets topic and Second is using kafka api.

It seems using kafka broker api is much more accurate, if cluster has many consumers - which means producing lots of __consumer_offsets, creating lag for consuming them and shows totally inaccurate lag monitoring.
The only disadvantage seems sending api intervally can be burdensome to broker, but it looks very slight.

I have no idea why the most famous lag monitoring systems like burrow, kafka manager use this inaccurate method.

david
  • 1
  • 3
  • What is your question? Remember, "Why" questions usually tend to get closed because they are mostly answered opinion-based. – Michael Heil Apr 16 '21 at 06:17
  • Also, questions seeking recommendations for tools are similarly opinion based - [help], but polling the "api" would not be the most accurate because there's a delay... Consuming the offsets topic doesn't cause any more problems than consuming any other topic, and will be more real-time. AFAIK, CMAK doesn't poll the offsets topic – OneCricketeer Apr 17 '21 at 20:08

0 Answers0