0

My microservice uses confluent-kafka-python. Once in a while it fails with this error

%4|1654121013.314|MAXPOLL|rdkafka#consumer-1| [thrd:main]: Application maximum poll interval (300000ms) exceeded by 67ms (adjust max.poll.interval.ms for long-running message processing): leaving group

Whenever it hits this error, it goes into idle instead of terminating.

Snippet of the consumer code:

    def start(self, callback: Callable[[List[Message]], None]) -> None:
        '''
        start consuming
        '''

        try:
            while self.__terminate_event is None or not self.__terminate_event.is_set():

                if not self.__consumer.assignment():
                    self.__consumer.subscribe(
                        [self.__topic],
                        on_assign=_on_assign,
                        on_revoke=_on_revoke,
                        on_lost=_on_lost
                    )

                    log.info("subscribed to topic: %s", self.__topic)

                message_list = self.__consumer.consume(
                    num_messages=self.__num_messages_per_poll, timeout=KAFKA_TIMEOUT
                )

How can it capture this error, so it terminates and let another service respawn?

Thanks

wxh
  • 619
  • 7
  • 20

0 Answers0