Why does pickle.dumps call __getattr__?

Question

import cPickle

class Foo(object):
    def __init__(self):
        self._data = {'bar': 'baz'}

    def __getattr__(self, name):
        assert hasattr(self, '_data')
        return self._data[name]

    # I even had to define this just to stop KeyError: '__getstate__'
    def __getstate__(self):
        return self.__dict__

foo = Foo()
bar = cPickle.dumps(foo)
cPickle.loads(bar)

This raises an assertion error.

I thought pickle/cPickle just turns __dict__ into a string when dumping and then uses that string to set the __dict__ of the new object directly when loading. Why would dumps need to call bar.__getattr__? How can I change Foo to avoid that?

Answer 1

According the documentation for cPickle: http://docs.python.org/library/pickle.html

object.__getstate__()

Classes can further influence how their instances are pickled; if the class defines the method __getstate__(), it is called and the return state is pickled as the contents for the instance, instead of the contents of the instance’s dictionary. If there is no __getstate__() method, the instance’s __dict__ is pickled.

Note

At unpickling time, some methods like __getattr__(), __getattribute__(), or __setattr__() may be called upon the instance. In case those methods rely on some internal invariant being true, the type should implement either __getinitargs__() or __getnewargs__() to establish such an invariant; otherwise, neither __new__() nor __init__() will be called.

Since you are trying to assert that hasattr(self, '_data') is True, I believe that you need to use __getinitargs__() or __getnewargs__(). This is because when using pickle, a classes __init__ method is not called.