19

We recently updated our Kafka brokers and clients to 1.1.1. Since the upgrade we periodically see INFO log entries such as

INFO Jun 08 08:30:20.335 61161458 [KafkaRecordConsumer-0] org.apache.kafka.clients.FetchSessionHandler [Consumer clientId=consumer-1, groupId=group_60_10] Node 3 was unable to process the fetch request with (sessionId=819759315, epoch=145991): INVALID_FETCH_SESSION_EPOCH.

I see that this message comes from the changes introduced in KIP-227: Introduce Incremental FetchRequests To Increase Partition Stability. However, I cannot find any detailed information about why this message would appear or what parameters might have to be tuned after its introduction.

So far it doesn't seem to have an impact on consumer behaviour with respect to receiving records (except introducing additional latency) but I would like to understand

  1. Why is the message being logged?
  2. What can be done to stop it being logged?
Mark
  • 28,783
  • 8
  • 63
  • 92
  • I have observed the above error in case I have multiple partition for topics and it does impact by consumption rate. I have 2 topics each having 4 partitions and 4 consumer threads at logstash end. Couple of my logstash consumer threads stopped processing messages with error `INVALID_FETCH_SESSION_EPOCH` – Krishna Oza Jul 10 '18 at 11:36
  • @darth_coder I posted to the Kafka mailing list with DEBUG logging but no response so far http://mail-archives.apache.org/mod_mbox/kafka-users/201808.mbox/%3CCAPiU9bWqARqBv3MjR9xYvZ69f6Zom1y5fbcatCHDMWoUB8YGTQ%40mail.gmail.com%3E – Mark Aug 02 '18 at 20:20
  • Mark, better to put the query on https://discuss.elastic.co/c/logstash – Krishna Oza Aug 03 '18 at 04:45
  • @darth_coder its fundamentally a Kafka issue. Not related to logstash. – Mark Aug 07 '18 at 16:39
  • Is your delivery configured to be exactly once? Have you configured your producer to `enable.idempotence=true`? – ErvalhouS Mar 25 '19 at 11:45
  • @ErvalhouS No I'm not using exactly once. – Mark Mar 25 '19 at 11:49
  • Someone filed a possibly related bug/question recently https://issues.apache.org/jira/browse/KAFKA-8052 – Mark Apr 01 '19 at 19:28
  • There is also a bug in Sarama, I've just had a similar case: https://stackoverflow.com/questions/62454882/how-to-scale-max-incremental-fetch-session-cache-slots – beck Jun 30 '20 at 16:57

1 Answers1

3

This was a race condition in Kafka - KAFKA-8052.

It will be fixed in the 2.3.0 release.

Mark
  • 28,783
  • 8
  • 63
  • 92