Pyro4: Deserializing protobuf class

Question

I am completely new to Pyro4. I wish to serve a python class which includes an attribute which is an instance of a protobuf object compiled to a python class.

import sys
import os
import Pyro4
import MPB_pb2 as mpb  # My Protobuf class

@Pyro4.expose
@Pyro4.behavior(instance_mode="single")
class MyWrappedProtobuf(object):
    def __init__(self):
        self.setup_MyProtobuf()

    def setup_MyProtobuf(self):
        self.__MyProtobuf = mpb.MyProtobufMessage()

    @property
    def MyProtobuf(self):
        return self.__MyProtobuf

After setting the server running, I create a proxy of the served object, and query the MyProtobuf property:

u = Pyro4.Proxy('PYRONAME:{}'.format(<server name>)).MyProtobuf

What I get back is a serialized object:

>>>u
{'serialized': '\n$\n\x05world\x12\x1b\t\x00\x00\x00\x00\x00\x00\x00\x00\x11\x00\x00\x00\x00\x00\x00\x00\x00\x19\x00\x00\x00\x00\x00\x00\x00\x00'}

I understand from the documents (https://pythonhosted.org/Pyro4/clientcode.html#changing-the-way-your-custom-classes-are-de-serialized) that "By default, custom classes are serialized into a dict. They are not deserialized back into instances of your custom class."

I am looking for instruction, or an example describing how I can go about turning this serialized dictionary back into my class, including the protobuf message instance that class contains.

It seems like this must be a common, and trivial operation, and yet I can't find a good example to show me, and the documentation is not providing explicit help that I can see.

Thank you!

score 1 · Accepted Answer · answered Jan 14 '19 at 20:03

By default, Pyro uses the Serpent serializer, which in turn uses python's primitive types to serialize stuff into. Usually this will be a dictionary. If it's a class it doesn't readily recognize or the class doesn't have a "suitable" serialization method defined, it will fall back to a special dictionary form such as this:

{
    "__class__":   "pakcage.Classname"
    "property1":   "value1",
    ...
}

You're not seeing this in your serialized form. Which means it wasn't Pyro (or Serpent, rather) that serialized it for you. What happened (I think) is that a Protobuf object defines a __getstate__() method that returns the serialized form of the protobuf object that you're seeing. The reverse of this is __setstate__(...) (these methods are 'borrowed' from Python's built in pickle serializer mechanism). I don't have experience with protobufs but my guess is that a simple:

u = proxy.MyProtoBuf
message = mpb.MyProtobufMessage()   # create empty protobuf object (??)
message.__setstate__(u)

will do the trick. I advise you to look into protobuf's documentation about how their objects are serialized (you may have to search for how they are pickled). Bottom line; this is not something Pyro (or serpent) has control over.

score 0 · Answer 2 · answered Jan 15 '19 at 20:45

It was indeed serializing from protobuf which I was seeing. Once that confucion was sorted, I found that the serialized results were unicode. In order for me to be able to get protobuf to deserialize the results, I needed to encode it as pure ascii (using <result>.encode('utf-8')). I'm not sure why it's returning unicode which it won't thereafter turn around and accept. However, I have a working solution.

Pyro4: Deserializing protobuf class

2 Answers2