Serialize/Deserialize simple but nested object to/from JSON

Question

I'm trying to transmit a Config from client to server.

• Config contains a password key I must NOT transmit
• Config contains a couple of simple objects, which are just key/value pairs (with value being a basic primitive)

This code works:

class Empty:
    pass

class Config:
    def __init__(self):
        # don't want to transmit this over the internet
        self.secret = 'P@ssw0rd'

    def create(self, foo):
        self.foo = foo  # property passed in
        self.bar = f'Hello {foo}'  # calculated property

        # A couple of custom objects, but they are simple
        # (only containing key/value pairs where value is basic primitive)
        self.v = Empty()
        self.v.a = 1

        self.w = Empty()
        self.w.b = 2

    def export_json(self):
        J = {}
        for k, v in vars(self).items():
            if k == 'secret':
                continue
            J[k] = vars(v) if isinstance(v, Empty) else v
        return J

    def construct_from_json(self, J_str):
        J = json.loads(J_str)
        for k, v in J.items():
            if isinstance(v, dict):
                _ = Empty()
                for k_, v_ in v.items():
                    setattr(_, k_, v_)
                v = _
            setattr(self, k, v)

Test:

```python
c = Config()
c.create('123')

J = c.export_json()
print('Serialized:')
print(json.dumps(J, indent=4))

d = Config()
d.construct_from_json(J)
print('Reconstructed: w.b = ', d.w.b)

Output:

Serialized:
{
    "foo": "123",
    "bar": "Hello 123",
    "v": {
        "b": 2
    },
    "w": {
        "b": 2
    }
}
Reconstructed: w.b =  2

However, is there a preferred/pythonic way to do this?

Answer 1

As someone mentioned in the comments, you might just want to use the pickle library here to avoid having to serialize/deserialize yourself, and to avoid having to make major modifications to the serialization code in the future if you add nested structures / etc or want to ignore other attributes. Here's a version of your code that works with pickle, while not serializing the secret attribute.

class Empty:
    pass

class Config:
    def __init__(self):
        # don't want to transmit this over the internet
        self.secret = 'P@ssw0rd'

    def create(self, foo):
        self.foo = foo  # property passed in
        self.bar = f'Hello {foo}'  # calculated property

        # A couple of custom objects, but they are simple
        # (only containing key/value pairs where value is basic primitive)
        self.v = Empty()
        self.v.a = 1

        self.w = Empty()
        self.w.b = 2
    
    # This gets the state for pickling. Note how we are explicitly removing
    # the `secret` attribute from the internal dictionary. You don't need to
    # do anything else
    def __getstate__(self):
        state = self.__dict__.copy()
        del state['secret']
        return state

Testing it out:

import pickle
c = Config()
c.create('123')

J = pickle.dumps(c)
print("Serialized: ", J)

d = pickle.loads(J)
print("Reconstructed w.b:", d.w.b)
print("Reconstructed secret:", d.secret)

And this is the output it produces (as wanted):

Serialized:  b'\x80\x04\x95m\x00...(truncated)'
Reconstructed w.b: 2
Traceback (most recent call last):
  File "/Users/mustafa/scratch/test.py", line 36, in <module>
    print("Reconstructed secret:", d.secret)
AttributeError: 'Config' object has no attribute 'secret'