Kafka consumer offset max value?

Question

I was googling and reading Kafka documentation but I couldn't find out the max value of a consumer offset and whether there is offset wraparound after max value. I understand offset is an Int64 value so max value is 0xFFFFFFFFFFFFFFFF. If there is wraparound, how does Kafka handle this situation?

score 49 · Accepted Answer · answered Dec 22 '15 at 21:58

49

According to this post, the offset is not reset:

We don't roll back offset at this moment. Since the offset is a long, it can last for a really long time. If you write 1TB a day, you can keep going for about 4 million days.

Plus, you can always use more partitions (each partition has its own offset).

So as Luciano said, probably not worth worrying about.

answered Dec 22 '15 at 21:58

Matt Hulse

5,496
4
29
37

25

Yup. Just like Y2K :) – Nick Nov 04 '17 at 23:38
I think we'll have to worry about it in the following 10 years. – rodolk Dec 10 '19 at 06:32

score 4 · Answer 2 · answered Dec 22 '15 at 19:57

4

It seems that this is not really "handled". But, taking into account that the offset is per partition, it seems this is something we should not worry about :)

Please see http://search-hadoop.com/m/uyzND1uRn8D1sSH322/rollover/v=threaded

answered Dec 22 '15 at 19:57

Luciano Afranllie

4,053
25
23

Kafka consumer offset max value?

2 Answers2

Linked