Kafka producer using python: TypeError: all produce message payloads must be null or type bytes

Question

I just started learning Python and Kafka. This is the first example I tried to get started. http://www.giantflyingsaucer.com/blog/?p=5541

And I got an exception:

Traceback (most recent call last):
  File "producer.py", line 23, in <module>
    main()
  File "producer.py", line 18, in main
    print_response(producer.send_messages(topic, msg))
  File "D:\Setups\Python35-32\lib\site-packages\kafka\producer\simple.py", line 50, in send_messages
    topic, partition, *msg
  File "D:\Setups\Python35-32\lib\site-packages\kafka\producer\base.py", line 379, in send_messages
    return self._send_messages(topic, partition, *msg)
  File "D:\Setups\Python35-32\lib\site-packages\kafka\producer\base.py", line 396, in _send_messages
    raise TypeError("all produce message payloads must be null or type bytes")
TypeError: all produce message payloads must be null or type bytes

I've searched on google but I'm not quite sure what the problem is. Could anyone please give me some advice? Thank you very much!

Here is my code:

from kafka import SimpleProducer, KafkaClient 

def print_response(response=None):
    if response:
        print('Error: {0}'.format(response[0].error))
        print('Offset: {0}'.format(response[0].offset))


def main():
    kafka = KafkaClient("10.2.5.53:9092")
    producer = SimpleProducer(kafka)

    topic = 'test'
    msg = 'Hello World'

    print_response(producer.send_messages(topic, msg))

    kafka.close()

if __name__ == "__main__":
    main()

My guess would be that the error you're getting has to do with closing the kafka client before the producer actually responds, since it is an asynchronous process. — jimijazz, Oct 20 '16 at 02:01
Take note that the Simple Producer and consumer APIS are deprecated as noted [here] (https://github.com/dpkp/kafka-python) (and they are simple implementations, but more complex to use!). You should use the high-level produced mentioned in that same page instead. — jimijazz, Oct 20 '16 at 02:02

score 0 · Answer 1 · edited May 23 '17 at 12:24

0

Oh, I just realized from looking at the example you posted is that strings must be byte strings, prepended with a b character, as explained here

Also, you're missing the try-except clause catches exceptions for when the server is not ready...

edited May 23 '17 at 12:24

Community

1
1

answered Oct 20 '16 at 02:26

jimijazz

2,197
1
15
24

Kafka producer using python: TypeError: all produce message payloads must be null or type bytes

1 Answers1