cross language (de)serialization

Question

i am trying to serialize in python and unmarshal in golang but i am facing error.

error message -- "cannot parse invalid wire-format data".

code configuration --

python code --

schema_registry_client = SchemaRegistryClient({'url': 'http://localhost:8082'})
protobuf_serializer = ProtobufSerializer(user_attributes_pb2.UserProperties,
                                           schema_registry_client,
                                           {'use.deprecated.format': True})
  producer_conf = {'bootstrap.servers':  'localhost:9092', 'key.serializer': StringSerializer('utf_8'), 'value.serializer': protobuf_serializer}
  producer = SerializingProducer(producer_conf)
  producer.poll(0.0)
  ########## Add an address #########
  PromptForAddress(user_attr)
  producer.produce(topic=topic, key=str(uuid4()), value=user_attr)
  producer.flush()

golang code --

        Brokers: []string{brokerAddress},
        Topic:   topic,
        GroupID: "test-consumer-group",
        Logger: l,
    })
    for {
        msg, err := r.ReadMessage(ctx)
        user := &pb.UserProperties{}
        err = proto.Unmarshal(msg.Value, user)
        // client.Query() NewKey(Namespace, Set, user.Id)
        if err != nil {
            log.Fatal(err)
        }
        fmt.Printf("\n%s\n", proto.Message(user))
        fmt.Printf("\n%v\n", user)
        if err != nil {
            panic("could not read message " + err.Error())
        }
        // after receiving the message, log its value
        fmt.Println("received: ", string(msg.Value))
    }

proto file is similar in both languages.

score 0 · Accepted Answer · answered Sep 07 '22 at 13:58

Since you're using Confluent Python serializer with their Schema Registry, you need to do the same in Go, and cannot just use plain Protobuf deserialization.

Confluent also maintains a Go client which you can use. Example for Protobuf - https://github.com/confluentinc/confluent-kafka-go/blob/master/examples/protobuf_consumer_example/protobuf_consumer_example.go

I'm not sure what use.deprecated.format does for Python, but Go might not accept that

cross language (de)serialization

1 Answers1