0

How to pickle Python class method types? It is not implemented in standard library. I found that I can use some third party modules like dill but how to do it with pickle only.

I prepared some test code to make this question simple:

import pickle

class Thing(object):
  @staticmethod
  def static_method():
    print Thing.static_method.__name__

  @classmethod
  def class_method(cls):
    print cls.class_method.__name__

  def object_method(self):
    print self.object_method.__name__


def main():
  items = [Thing,
           Thing(),
           Thing.static_method,
           Thing.class_method,
           Thing.object_method,
           Thing().object_method]
  for i, item in enumerate(items):
    try:
      pickle.loads(pickle.dumps(item))
    except Exception as ex:
      print i, ex, item

if __name__ == '__main__':
  main()

How to deal with default lack of pickle support for methods in Python 2.7?

2 Can't pickle <function static_method at 0x025614F0>: it's not found as __main__.static_method <function static_method at 0x025614F0>
3 can't pickle instancemethod objects <bound method type.class_method of <class '__main__.Thing'>>
4 can't pickle instancemethod objects <unbound method Thing.object_method>
5 can't pickle instancemethod objects <bound method Thing.object_method of <__main__.Thing object at 0x024F71D0>>
DNA
  • 42,007
  • 12
  • 107
  • 146
Chameleon
  • 9,722
  • 16
  • 65
  • 127
  • 1
    Have you seen this very similar question?: http://stackoverflow.com/questions/9470403/pickling-class-method – ajdigregorio Feb 20 '15 at 00:24
  • 2
    I'm the `dill` author. Why stick with `pickle` when `dill` can pickle all of your `items`? – Mike McKerns Feb 20 '15 at 03:47
  • 1
    If it is because you want to use `multiprocessing`, then you can just use my fork of `multiprocessing` called `pathos.multiprocessing`. It uses the `dill` serializer, so you can send any of your `items` in a `map` function. For example, see: http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemethod-when-using-pythons-multiprocessing-pool-ma/21345273#21345273 – Mike McKerns Feb 20 '15 at 03:48
  • @MikeMcKerns I feel that dill is too big to solve this problem (loading cost + wasted memory) - I can pick some code from dill and use but not want load all since only some function is required - I think that will be 1% of 100%. – Chameleon Feb 20 '15 at 08:28
  • @paintedcones I read answer but not found useful - I found some better answers but without solving problem of `@staticmethod`. – Chameleon Feb 20 '15 at 08:34
  • @MikeMcKerns I will test and try `dill` but not think that is good because my restrictions and not sure if it Pure python - it is great work whatever code need be little cleared/documented to make this library more popular that is my impression at first sight. – Chameleon Feb 20 '15 at 08:43
  • @MikeMcKerns I do some test and it generates about 7M + 0.45s footprint - time is o.k. and memory usage is high whatever it is o.k. for many applications https://www.dropbox.com/s/bjrkxw2sfzwfuns/Zrzut%20ekranu%202015-02-20%2009.57.33.png?dl=0 – Chameleon Feb 20 '15 at 09:13
  • It's pure python. I am also very aware of keeping the load times down, as I primarily use `dill` as a means for enabling high-performance parallel and distributed computing in python. The memory usage is a bit high *initially* b/c it builds and registers (in the `pickle` registry) a minimal version of many of the objects in the standard library. Additional memory consumed after import is minimal. – Mike McKerns Feb 20 '15 at 12:55
  • @MikeMcKerns It is very good that is pure Python. I've learning library. I think that load tme is o.k. but memory footprint is high for some uses (If have high scalable server with thread limited to 128M to serve more users with excepted response 1s) and no problem if you use it on PC. I prefer libraries with load on demand since if you join many things with initialization footprint it will sum over 1s that is not problem in science application at all or mass processing. – Chameleon Feb 20 '15 at 15:39
  • @MikeMcKerns Can I extract some code from your library and for lightweight version of dill and use it in commercial application? I can share code via GitHub if it need or report some bugs to dill. Whatever I need lightweight library to speed up code. Is your license support it? – Chameleon Feb 20 '15 at 17:24
  • It's a BSD license. Have at it. It's already used commercially. Please do report bugs and share code via github. – Mike McKerns Feb 20 '15 at 18:12
  • @Mike McKerns Only bug which I observer is not handling __class_mangled what is marked for fixme I looks that is easy to fix. I found that I will make multiprocessing simpler using callback so I try map/task/reduce pattern - so class will call some global def using `self` as parameter - it will allow good inheritance and code will be light what I like. – Chameleon Feb 22 '15 at 13:47

0 Answers0