Questions tagged [dill]

Dill is a module which extends python's 'pickle' module for serializing and de-serializing python objects to the majority of the built-in python types. Use this tag in conjunction with the pickle and python tag for questions about object serialization with Dill.

Dill extends python's 'pickle' module for serializing and de-serializing python objects to the majority of the built-in python types.

Major features

Dill can pickle the following standard types:

  • none, type, bool, int, long, float, complex, str, unicode,
  • tuple, list, dict, file, buffer, builtin,
  • both old and new style classes,
  • instances of old and new style classes,
  • set, frozenset, array, functions, exceptions

Dill can also pickle more 'exotic' standard types:

  • functions with yields, nested functions, lambdas
  • cell, method, unboundmethod, module, code, methodwrapper,
  • dictproxy, methoddescriptor, getsetdescriptor, memberdescriptor,
  • wrapperdescriptor, xrange, slice,
  • notimplemented, ellipsis, quit

Dill cannot yet pickle these standard types:

  • frame, generator, traceback

Dill also provides the capability to:

  • save and load python interpreter sessions
  • save and extract the source code from functions and classes
  • interactively diagnose pickling errors

Current Release

The latest released version of dill is available from http://dev.danse.us/trac/pathos and https://github.com/uqfoundation/dill.

Dill is distributed under a 3-clause BSD license.

290 questions
2
votes
2 answers

Get a function pickleable for using in Differential Evolution workers = -1

#I EDITED MY ORIGINAL POST in order to put a simpler example. I use differential evolution (DE) of Scipy to optimize certain parameters. I would like to use all the PC processors in this task and I try to use the option "workers=-1" The codition…
2
votes
0 answers

Pathos p_tqdm multiprocess error with dill

I'm trying to run some code through the p_tqdm library with p_map() to parallelize some code. I run into this dill-related error that I can't figure out. Traceback (most recent call last): File…
Antoine Zambelli
  • 724
  • 7
  • 19
2
votes
0 answers

Python dill : KeyError: 'ClassType'

I'm trying to pickle a class. Pickle fails with attribute error because the class constructor is not present at load time. I gathered that dill module is an extension of pickle module. However, I still get errors. Namely: Traceback (most recent call…
Alex Deft
  • 2,531
  • 1
  • 19
  • 34
2
votes
1 answer

How to dump a function with import inside

I am trying to dump functions into a file, so that I can use this file/function elsewhere. I choose dill rather than pickle, because I need dependencies. However, dill doesn't work if the function has imports inside. For example: def func(): …
small
  • 25
  • 5
2
votes
0 answers

Serialize subclass of abstract class in python 3.7.1

I have the following script: from abc import ABC, abstractmethod import dill class A(ABC): @abstractmethod def test(self, a): """Abstract method""" class B(A): def test(self, a): return a + 1 if __name__ ==…
hchw
  • 1,388
  • 8
  • 14
2
votes
2 answers

Multiprocessing using Pool class in Python giving Pickling error

I am trying to run a simple multiprocessing example in python3.6 in a zeppelin notebook(in windows) but I am not able to execute it. Below is the code that i used: def sqrt(x): return x**0.5 numbers = [i for i in range(1000000)] with Pool() as…
Shubham Kedia
  • 77
  • 1
  • 5
2
votes
1 answer

Dynamic inheritance that is pickleable?

After much searching, the only way I have found to solve my particular problem is to use dynamic inheritance. It is easy enough following the guide from here and a few other SO questions; most table is this. Using a modified version of the contrived…
Justin Winokur
  • 91
  • 1
  • 1
  • 7
2
votes
0 answers

How to pickle python object which has reference to cytpes library

I am trying to pickle python function which uses ctypes object to call some function. I have created a '.so' file with my sum function which I am trying to invoke in my python code c code #include int sum(int a, int b) { return a +…
2
votes
1 answer

How to load a Spacy object (pickled in Ubuntu) on a Windows machine?

I use a Ubuntu machine for developing code and then use Windows for deployment. So, I have pickled a Spacy Vectorizer object using dill on my Ubuntu machine. Now I am trying to load it back on a Windows machine (un-pickle it) but I am getting this…
Pranzell
  • 2,275
  • 16
  • 21
2
votes
1 answer

how to make dill load python2 pickle in python3

If pickled using pickle then following snippet works. but if I have an object dumped using dill, dill.load does not work because dill.load does not accept any encoding argument. is there any way to make this work using dill? with…
muon
  • 12,821
  • 11
  • 69
  • 88
2
votes
1 answer

dill.dump_session not working with getpass

I have a Jupyter notebook whose state I want to save using dill.dump_session, however the function throws an error if the notebook uses the getpass module. Is there any way around this issue? I understand the point of not dumping the password, but I…
2
votes
0 answers

error KeyError: 'ObjectType' when loading pickle

I'm using python 3 on mac. I have tried to load a pickle into python and i saw an error regarding my "dill" package. after a lot of reading i found a specific version of dill that works for python2 -pip install dill==0.2.7.1. my question is what…
Yuval Cohen
  • 131
  • 1
  • 5
2
votes
1 answer

python 3: error when dump/load sympy lambdify-epression

I want to save/serialize a Sympy-Lambdify func into a file and use/load it by another python-program later. Case 1: it works well import dill import sympy as sp from sympy.utilities.lambdify import lambdify dill.settings['recurse'] = True a,b =…
sun0727
  • 374
  • 4
  • 19
2
votes
0 answers

Class inheritance type checking after pickling in Python

Is there a sure-fire way to check that the class of an object is a sub-class of the desired super? For Example, in a migration script that I'm writing, I have to convert objects of a given type to dictionaries in a given manner to ensure two-way…
Minothor
  • 316
  • 4
  • 15
2
votes
0 answers

Unable to obtain results from multiprocessing with pooling

Firstly apologies for any mistakes I make. I am new to StackOverflow and I'm still getting used to it. The Background: I try to train multiple keras models via multiprocessing. Those Keras models are stored in three lists (currently 50 models per…
Lennart S.
  • 91
  • 6