2

What happens to message with null key when log compaction is enabled? Are messages with a null key compacted?

Cr4zyTun4
  • 625
  • 7
  • 18

1 Answers1

2

In current version what happens is this:

"Currently when a user sends a null key to a log compacted topic, the broker returns CorruptRecordException, which is a retriable exception. As such, the producer keeps retrying until retries are exhausted or request.timeout.ms expires and eventually throws a TimeoutException."

As reported in Jira KIP-135 this is a bit confusing.

In the future if the cleaner solution from KIP-135 (still under discussion as @Ran Lupvoich pointed out) get accepted another behavior should be the following:

When you try to produce null key to log compacted topic, Kafka should return a new error type INVALID_KEY

Cr4zyTun4
  • 625
  • 7
  • 18
nipuna
  • 3,697
  • 11
  • 24
  • 1
    Notice the status, as it seems not yet merged to master. Status Current state: "Under Discussion" Discussion thread: here JIRA: KAFKA-4808 - send of null key to a compacted topic should throw error back to user PATCH AVAILABLE – Ran Lupovich Jun 08 '21 at 13:06
  • So what happens right now is this: "Currently when a user sends a null key to a log compacted topic, the broker returns CorruptRecordException, which is a retriable exception. As such, the producer keeps retrying until retries are exhausted or request.timeout.ms expires and eventually throws a TimeoutException." Correct? – Cr4zyTun4 Jun 08 '21 at 13:09
  • 1
    Yes. And that error should be updated after that fix done I mentioned in the answer – nipuna Jun 08 '21 at 13:11