0

I want to handle the case if Kafka Broker is down on the Kafka Producer end then its taking longer time to show the below error.

Failed to send; nested exception is org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for logging-0: 30030 ms has passed since batch creation plus linger time

How to handle this?

BalusC
  • 1,082,665
  • 372
  • 3,610
  • 3,555
Abhishek Mittal
  • 91
  • 2
  • 7

1 Answers1

2

The producer waits for request.timeout.ms for a response from the broker.

The configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted. This should be larger than replica.lag.time.max.ms (a broker configuration) to reduce the possibility of message duplication due to unnecessary producer retries.

It is set to 30000ms by default. Be careful if you try reducing it as if too short it can cause the producer to retry too quickly and produce duplicates.

Mickael Maison
  • 25,067
  • 7
  • 71
  • 68
  • I cant find replica.lag.time.max.ms in server.properties can you pls tell how to configure this in broker – Abhishek Mittal Apr 11 '18 at 10:49
  • See it in the broker configuration documentation: http://kafka.apache.org/documentation/#brokerconfigs – Mickael Maison Apr 11 '18 at 11:35
  • 1
    Also see `max.block.ms` [in this answer](https://stackoverflow.com/questions/49487619/how-to-set-timeout-for-onfailure-event-spring-kafka/49494338#49494338) which can affect getting metadata when the app is first started and the broker is down. – Gary Russell Apr 11 '18 at 13:41