0

I want to be able to read messages that come in specific partitions of a topic and also messages in another topic like a I would with a simple Consumer.

self.consumer = AvroConsumer(conf)

parts = [TopicPartition('p_topic', 13),
         TopicPartition('p_topic', 14)

self.consumer.assign(parts)
self.consumer.subscribe(['test_topic'])

some 'clients' produce messages in partitions of 'p_topic', and some (the ones I created) in 'test_topic' like this:

self.p.produce('test_topic', msg)

I cannot integrate these two though with the code shown above. The messages I produce in 'test_topic' throw:

File "/usr/local/lib/python2.7/dist-packages/confluent_kafka/avro/__init__.py", line 115, in poll
    decoded_value = self._serializer.decode_message(message.value())
  File "/usr/local/lib/python2.7/dist-packages/confluent_kafka/avro/serializer/message_serializer.py", line 214, in decode_message
    raise SerializerError("message does not start with magic byte")
SerializerError

How can I read both at the same time using the AvroConsumer?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
ealiaj
  • 1,525
  • 1
  • 15
  • 25

1 Answers1

0

According to the error about "magic byte", whatever was produced into that topic was not done with AvroProducer.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245