I'm using the multiprocess library (note, not multiprocessing), in a notebook along with nbdev. When I change and save one of the modules in the project, which is then imported in the notebook using multiprocess, I get this error message and the processes crash:
/home/user/code/env/lib/python3.8/site-packages/dill/_dill.py:1890: PicklingWarning: Pickling a PyCapsule (b'datetime.datetime_CAPI') does not pickle any C data structures and could cause segmentation faults or other memory errors when unpickling.
warnings.warn('Pickling a PyCapsule (%s) does not pickle any C data structures and could cause segmentation faults or other memory errors when unpickling.' % (name,), PicklingWarning)
Process SpawnPoolWorker-8:
Traceback (most recent call last):
File "/home/user/code/env/lib/python3.8/site-packages/multiprocess/process.py", line 315, in _bootstrap
self.run()
File "/home/user/code/env/lib/python3.8/site-packages/multiprocess/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/home/user/code/env/lib/python3.8/site-packages/multiprocess/pool.py", line 114, in worker
task = get()
File "/home/user/code/env/lib/python3.8/site-packages/multiprocess/queues.py", line 361, in get
return _ForkingPickler.loads(res)
File "/home/user/code/env/lib/python3.8/site-packages/dill/_dill.py", line 286, in loads
return load(file, ignore, **kwds)
File "/home/user/code/env/lib/python3.8/site-packages/dill/_dill.py", line 272, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File "/home/user/code/env/lib/python3.8/site-packages/dill/_dill.py", line 419, in load
obj = StockUnpickler.load(self)
File "/home/user/code/env/lib/python3.8/site-packages/fastcore/basics.py", line 877, in __getattr__
if not self.ready:
File "/home/user/code/env/lib/python3.8/site-packages/fastcore/basics.py", line 877, in __getattr__
if not self.ready:
File "/home/user/code/env/lib/python3.8/site-packages/fastcore/basics.py", line 877, in __getattr__
if not self.ready:
[Previous line repeated 2986 more times]
RecursionError: maximum recursion depth exceeded
This is somewhat generic, alluding to perhaps a problem with pickling datetime.datetime_CAPI objects. How can I start to find the source of the problem?