I have set my segment retention via (log.retention.ms) for topic to 2h. However I still have a lot of old messages in the message queue and some of the segment data files (data/topic-0/00000000000000658568.log) are older than two months. I was expecting kafka to roll these over, but it doesn't. Why?
Asked
Active
Viewed 37 times
0
-
The [`log.retention.[hours|mintes|ms]` directives](https://www.conduktor.io/kafka/kafka-topic-configuration-log-retention/) specify a minimum retention time for messages, in other words the amount of time after which older messages ***may*** be deleted. They don't have the effect that Kafka ***will*** delete messages *immediately* after that time has expired. - *" Time limit can be much greater than expected if the segment is very big (few messages per day in a 1GB segment)."* https://www.conduktor.io/kafka/kafka-topic-configuration-log-retention/ – HBruijn Apr 21 '23 at 10:31