0

In our application there is a possibility that the message on Kafka topic is null, while the key is not empty. We'd like to handle those messages, but they are skipped instead of being consumed. Is there a possibility to consume a message from Kafka topic even when it is empty?

Isard
  • 312
  • 1
  • 14

1 Answers1

0

I doubt it, since there is no body it would not be deserialized. Unless you configured your own custom deserializer that treated a null message body as something other than null.

Chris Patterson
  • 28,659
  • 3
  • 47
  • 59
  • The thing is that the AvroDeserializer that we use supports deserializing null values since version 1.9.0. Only after switching to MassTransit the consumer stopped to consume messages. Here's the release notes: https://github.com/confluentinc/confluent-kafka-dotnet/releases/tag/v1.9.0 and here's the PR with given changes: https://github.com/confluentinc/confluent-kafka-dotnet/pull/1352. Maybe the support for this needs to be added in MassTransit as well? – Isard Oct 10 '22 at 13:37
  • You're welcome to review the changes and submit a PR. Beyond that, I don't have anything else to add. – Chris Patterson Oct 10 '22 at 13:53
  • Just to note that that scenario with none null key and null value is pretty common in Kafka world. Its called tombstone messages and indicate message deletion. It's used for topic compacting and GDRP reasons. – drakkar Oct 11 '22 at 14:07