Using getstate/setstate with pickle fails with "ValueError: size needs to be (int width, int height)"

Question

I am attempting to pickle the pygame.Surface object, which is not pickleable by default. What I've done is to add the classic picklability functions to the class and overwrite it. This way it will work with the rest of my code.

class TemporarySurface(pygame.Surface):
    def __getstate__(self):
        print '__getstate__ executed'
        return (pygame.image.tostring(self,IMAGE_TO_STRING_FORMAT),self.get_size())

    def __setstate__(self,state):
        print '__setstate__ executed'
        tempsurf = pygame.image.frombuffer(state[0],state[1],IMAGE_TO_STRING_FORMAT)
        pygame.Surface.__init__(self,tempsurf)

pygame.Surface = TemporarySurface

Here is an example of my traceback when I try to pickle a few recursive objects:

Traceback (most recent call last):
  File "dibujar.py", line 981, in save_project
    pickler.dump((key,value))
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 562, in save_tuple
    save(element)
  File "/usr/lib/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/usr/lib/python2.7/copy_reg.py", line 71, in _reduce_ex
    state = base(self)
ValueError: size needs to be (int width, int height)

The part that puzzles me is that the print statement is not being executed. Is __getstate__ even being called? I'm confused here, and I'm not exactly sure what information to put up. Let me know if anything additional would help.

It might not be called, depending on how different pieces of code import `Surface`. How is `Surface` used? Is it possible for you to use your subclass instead of monkeypatching `Surface` itself? — BrenBarn, Dec 07 '12 at 05:59
Well, I'm using other peoples code which passes around Surface objects, so it would be much more difficult for me to use the subclass. Surface is being used as normally and I only run into this error when attempting to pickle it. — hedgehogrider, Dec 07 '12 at 06:47
I found this: http://www.mail-archive.com/pygame-users@seul.org/msg13420.html However, are you able to retrieve the filepath or id, and save that instead of the actual pixels? — ninMonkey, Dec 08 '12 at 00:19

score 3 · Answer 1 · answered Dec 07 '12 at 07:47

As the documentation says, the primary entry point for pickling extension types is the __reduce__ or __reduce_ex__ methods. Given the error, it seems that the default __reduce__ implementation is not compatible with pygame.Surface's constructor.

So you'd be better off providing a __reduce__ method for Surface, or registering one externally via the copy_reg module. I would suggest the latter, since it doesn't involve monkey patching. You probably want something like:

import copy_reg

def pickle_surface(surface):
    return construct_surface, (pygame.image.tostring(surface, IMAGE_TO_STRING_FORMAT), surface.get_size())

def construct_surface(data, size):
    return pygame.image.frombuffer(data, size, IMAGE_TO_STRING_FORMAT)

construct_surface.__safe_for_unpickling__ = True
copy_reg.pickle(pygame.Surface, pickle_surface)

That should be all you need. Make sure that the construct_surface function is available at the top level of a module though: the unpickling process needs to be able to locate the function in order to perform the unpickling process (which might be happening in a different interpreter instance).

Using __getstate__/__setstate__ with pickle fails with "ValueError: size needs to be (int width, int height)"

1 Answers1

Using getstate/setstate with pickle fails with "ValueError: size needs to be (int width, int height)"