0

I am pretty new to the confluent-kafka and python, just would like to know if there a way in python we could serialize the python class to an kafka message using avro schema.

I am currently using AvroProducer provided by confluent-kafka, however, i am only able tot serialize a json string. And the downstream application is using Java, and downstream would like to deserialize the message back to an Java Object.

producer = AvroProducer(self.producer_config, default_key_schema=key_schema, default_value_schema=value_schema)

value = ast.literal_eval(str(message))

 try:
        producer.produce(topic=topic, partition=partition, key=str(message['Id']),
                         value=value)

I believe this is the part that is causing issue:

  value = ast.literal_eval(str(message))

The message is originally a dictionary. I have to convert it to a json string in order to serialize it. It just looks really weird.

Could you help advise how to serialie a python class, instead of serialize a json string using AvroProducer? Thanks!

Rachel
  • 349
  • 1
  • 3
  • 19
  • `import json; value = json.dumps(message)` there you go. – Albin Paul Jan 25 '22 at 14:32
  • I got an exception when try this 'str' object has no attribute 'get' – Rachel Jan 25 '22 at 14:52
  • you told message is dictionary, right? then why is there a string exception? message should be a dictionary for it to work. – Albin Paul Jan 25 '22 at 14:53
  • yes it is a diictionary, here is a sample message. {'LinkId': 'f7409a80c909d1407593d2a3bc31499d', 'Source': 'I', 'TradeId': 123, 'TransactionStatus': 'New'} – Rachel Jan 25 '22 at 15:33

0 Answers0