1

How to limit the event consumption speed of kafka consumer so that service is not impacted .Getting huge data in kafka Topic and i need to process all the events . We have 8 consumer reading from 12 partitions Getting huge data in kafka topic but retention is 5 days. So how can i limit from consumer side so that consumer won't go down. Is there any way in kafka to put a message reading speed from consumer side eg . like 100 events in 10 minutes .

Michael Heil
  • 16,250
  • 3
  • 42
  • 77

1 Answers1

0

You can limit consumer request using kafka quotas.

If your consumer is "going down" there may be other issues in the code. For example, not blocking requests for more messages until previous messages are processed completely. But quotas should help limit the consumers.

John Mercier
  • 1,637
  • 3
  • 20
  • 44