0

I'm using the multiprocess library (note, not multiprocessing), in a notebook along with nbdev. When I change and save one of the modules in the project, which is then imported in the notebook using multiprocess, I get this error message and the processes crash:

/home/user/code/env/lib/python3.8/site-packages/dill/_dill.py:1890: PicklingWarning: Pickling a PyCapsule (b'datetime.datetime_CAPI') does not pickle any C data structures and could cause segmentation faults or other memory errors when unpickling.
  warnings.warn('Pickling a PyCapsule (%s) does not pickle any C data structures and could cause segmentation faults or other memory errors when unpickling.' % (name,), PicklingWarning)
Process SpawnPoolWorker-8:
Traceback (most recent call last):
  File "/home/user/code/env/lib/python3.8/site-packages/multiprocess/process.py", line 315, in _bootstrap
    self.run()
  File "/home/user/code/env/lib/python3.8/site-packages/multiprocess/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/code/env/lib/python3.8/site-packages/multiprocess/pool.py", line 114, in worker
    task = get()
  File "/home/user/code/env/lib/python3.8/site-packages/multiprocess/queues.py", line 361, in get
    return _ForkingPickler.loads(res)
  File "/home/user/code/env/lib/python3.8/site-packages/dill/_dill.py", line 286, in loads
    return load(file, ignore, **kwds)
  File "/home/user/code/env/lib/python3.8/site-packages/dill/_dill.py", line 272, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "/home/user/code/env/lib/python3.8/site-packages/dill/_dill.py", line 419, in load
    obj = StockUnpickler.load(self)
  File "/home/user/code/env/lib/python3.8/site-packages/fastcore/basics.py", line 877, in __getattr__
    if not self.ready:
  File "/home/user/code/env/lib/python3.8/site-packages/fastcore/basics.py", line 877, in __getattr__
    if not self.ready:
  File "/home/user/code/env/lib/python3.8/site-packages/fastcore/basics.py", line 877, in __getattr__
    if not self.ready:
  [Previous line repeated 2986 more times]
RecursionError: maximum recursion depth exceeded

This is somewhat generic, alluding to perhaps a problem with pickling datetime.datetime_CAPI objects. How can I start to find the source of the problem?

Anonymous Entity
  • 3,254
  • 3
  • 27
  • 41

0 Answers0