0

Running

ccloud kafka topic consume <topic> -b --print-key

ends with error: panic: runtime error: index out of range [0] with length 0

What I am wondering is if this is something that can happen or am I looking at some kind of corrupt data in this topic?

Why I ask this, is because I am getting a null pointer exception in one of the services consuming a topic and I am not really sure why it happens since I would expect that even in the case of tombstone record key will not be null but value can be null (is null if tombstone record). So after filtering out messages that are null in value I am not sure what else to do?

[EDIT]: When I try the same thing with a confluent-kafka-python library then I get no error, but I do get some null records in the process. It seems that for now the error in code, when reading from a topic is solved with filtering out null values.

.filter((key, value) -> key != null || value != null)

I would have thought null keys are skipped?

[EDIT 2] Connected to my other, but related, problem not described here. I had no idea that .toStream() after a join with another KTable will push down the chain a tombstone record.

Dusko
  • 628
  • 2
  • 6
  • 24
  • Null keys are not skipped in Kafka Streams – OneCricketeer Nov 02 '20 at 23:58
  • @OneCricketeer ok, I understand they are not skipped if data is streamed, but for example in join they should be ignored? This issue that I have is related to the second issue where I try to do a join and end up with NPE where I would have expected null keys are ignored and I "only" have to filter out null value messages. – Dusko Nov 03 '20 at 09:47
  • A join on a stream assumes you have a non-null key, requiring it to be pre-filtered. If you are joining two tables, then you shouldn't have any null keys but you could still have null values – OneCricketeer Nov 03 '20 at 13:49
  • That I understand but what I had no idea is that after a join of two tables and then converting that result `toStream()` then I am passed a tombstone message also, and then that message will have `value == null`. I wasn't expecting I will get passed a tombstone message after I did a join on two tables. – Dusko Nov 03 '20 at 17:13
  • But what is still puzzling is why the above error when using `ccloud` and not implementation of `confluent-kafka-python` library. – Dusko Nov 03 '20 at 17:14
  • 1
    ccloud always tries to deserialize the bytearrays, I believe. The Python library will return None if the deserializer fails – OneCricketeer Nov 03 '20 at 21:03

0 Answers0