Questions tagged [dill]

Dill is a module which extends python's 'pickle' module for serializing and de-serializing python objects to the majority of the built-in python types. Use this tag in conjunction with the pickle and python tag for questions about object serialization with Dill.

Dill extends python's 'pickle' module for serializing and de-serializing python objects to the majority of the built-in python types.

Major features

Dill can pickle the following standard types:

  • none, type, bool, int, long, float, complex, str, unicode,
  • tuple, list, dict, file, buffer, builtin,
  • both old and new style classes,
  • instances of old and new style classes,
  • set, frozenset, array, functions, exceptions

Dill can also pickle more 'exotic' standard types:

  • functions with yields, nested functions, lambdas
  • cell, method, unboundmethod, module, code, methodwrapper,
  • dictproxy, methoddescriptor, getsetdescriptor, memberdescriptor,
  • wrapperdescriptor, xrange, slice,
  • notimplemented, ellipsis, quit

Dill cannot yet pickle these standard types:

  • frame, generator, traceback

Dill also provides the capability to:

  • save and load python interpreter sessions
  • save and extract the source code from functions and classes
  • interactively diagnose pickling errors

Current Release

The latest released version of dill is available from http://dev.danse.us/trac/pathos and https://github.com/uqfoundation/dill.

Dill is distributed under a 3-clause BSD license.

290 questions
8
votes
1 answer

multiprocessing -> pathos.multiprocessing and windows

I'm currently using the standard multiprocessing in python to generate a bunch of processes that will run indefinitely. I'm not particularly concerned with performance; each thread is simply watching for a different change on the filesystem, and…
jdmcbr
  • 5,964
  • 6
  • 28
  • 38
7
votes
2 answers

python - No module named dill while using pickle.load()

I have dill installed in my python 2.7 but when I try to unpickle my model it says "No module named dill". The pickled file contains pandas series. EDIT : Here's the snapshot of the traceback on ElasticBeanstalk environment File…
Ankush Bhatia
  • 133
  • 1
  • 1
  • 6
7
votes
1 answer

python pickle object with lambdas

How can I pickle a python object which contains lambdas? Can't pickle local object 'BaseDiscretizer.__init__..' is the error I get when trying to pickle…
Georg Heiler
  • 16,916
  • 36
  • 162
  • 292
7
votes
3 answers

Serializing an object in __main__ with pickle or dill

I have a pickling problem. I want to serialize a function in my main script, then load it and run it in another script. To demonstrate this, I've made 2 scripts: Attempt 1: The naive way: dill_pickle_script_1.py import pickle import time def…
Peter
  • 12,274
  • 9
  • 71
  • 86
7
votes
0 answers

Pickle a dynamically imported class

I have a bunch of objects created from classes imported with module = imp.load_source(packageName, packagePath) that I need to pickle. It all works perfectly, as long as packagePath is directly in the Python path or the working dir. But as soon as…
6
votes
1 answer

Loading a pkl file using dill

I have a very complex dictionary and dumping, loading directly using dill works. This is in reference to this answer. But there is a slight modification. I need to save this in some file and read that file for later use. Here's a piece of my…
Hypothetical Ninja
  • 3,920
  • 13
  • 49
  • 75
6
votes
1 answer

pandas.algos._return_false causes PicklingError with dill.dump_session on CentOS

I have a code framework which involves dumping sessions with dill. This used to work just fine, until I started to use pandas. The following code raises a PicklingError on CentOS release 6.5: import pandas import dill dill.dump_session('x.dat') The…
Korem
  • 11,383
  • 7
  • 55
  • 72
5
votes
0 answers

issue pickling weakrefs with dill

I am trying to implement classes for tree nodes that maintain a reference to their parent node. In a nutshell, whenever node A is assigned as attribute of node B, A's parent reference is automatically set to B. I also provide subclasses for tuples,…
harfel
  • 302
  • 3
  • 14
5
votes
2 answers

dill fails to pickle sympy.Float under Python 3.7 for dill.settings["recurse"]=True

Under python 3.7, dill fails to pickle sympy.Float objects when the "recurse" setting is set to True: dill.settings["recurse"] = True import sympy tst = sympy.Float(3.3) dill.detect.errors(tst) This fails with RecursionError('maximum recursion…
jaykay
  • 51
  • 2
5
votes
3 answers

What is the proper way to make an object with unpickable fields pickable?

For me what I do is detect what is unpickable and make it into a string (I guess I could have deleted it too but then it will falsely tell me that field didn't exist but I'd rather have it exist but be a string). But I wanted to know if there was a…
Charlie Parker
  • 5,884
  • 57
  • 198
  • 323
5
votes
2 answers

How can I recover a corrupted, partially pickled file?

My program was killed while serializing data (a dict) to disk with dill. I cannot open the partially-written file now. Is it possible to partially or fully recover the data? If so, how? Here's what I've tried: >>> dill.load(open(filename,…
eqzx
  • 5,323
  • 4
  • 37
  • 54
5
votes
2 answers

How to install dill in IPython?

Right at the outset, I tried conda install dill, and conda was not able to find it on the internet. Then I downloaded both .tgz and .zip files in my default IPython directory from here: https://pypi.python.org/pypi/dill After which I tried the…
Shashank Sawant
  • 1,111
  • 5
  • 22
  • 40
4
votes
0 answers

NameError after Dill/Marshal deserialisation

I am trying to dump a compiled piece of compiled Python code using either marshal or dill. The source is compiled with compiled(src, "", "exec", optimize=2), the compiled code does run well. However, when the compiled code is dumped with…
4
votes
1 answer

Dill dump_session: Making Jupyter-session pickle-able again

I like to use dill.dump_session to save my notebook state. However, I often have unpicklable objects loaded (dask-clusters, keras models). Is there any way I can remove them from my environment and then use dump_session()? I tried to find the…
JRein
  • 51
  • 4
4
votes
1 answer

Using dill to save current state of jupyter notebook

I have seen numerous answers stating to use dill for saving the current state of the kernel; import dill # save dill.dump_session('notebook_env.db') # load dill.load_session('notebook_env.db') Now, what is this notebook_env.db? Is it the path to…
the man
  • 1,131
  • 1
  • 8
  • 19
1
2
3
19 20