0

I am writing the following code:

from multiprocessing import Pool
import defs
variables = ['Temp'] # variables used for the clustering analysis

if __name__ ==  '__main__': 
    num_processors = 3
    p=Pool(processes = num_processors)
    output = list(p.imap(defs.f,variables))

where defs is a file that contains the function f which simply reads a netcdf file with xarray. But I get the error:'TypeError("cannot pickle '_thread.lock' object") in multiprocessing.pool(). I have Windows 10 and Python 3.9.12. Any help?

les2004
  • 105
  • 7
  • Does this answer your question? [Python multiprocessing PicklingError: Can't pickle ](https://stackoverflow.com/questions/8804830/python-multiprocessing-picklingerror-cant-pickle-type-function) – HTF Jun 02 '22 at 07:18
  • If `defs.f` is a function at global scope and is not doing anything that results in pickling, you shouldn't be having a problem. Are you sure xarray is not doing pickling? You need to post your `defs` module so we can see function `f` in context. And we need to see a stack trace so we know where you are getting this error. – Booboo Jun 02 '22 at 16:43
  • @HTF I don't thing this error is due to this `imap` call but rather something being done by `defs.f`. – Booboo Jun 02 '22 at 16:45

0 Answers0