1

I am trying to execute a python script (here called file_2.py) in parallel using ProcessPoolExecutor. The script uses functions I have written in another file (here called file_1.py). Both of these files are in the same directory.

If I important the functions and run the script sequentially, all is fine. Moreover, when I copy the functions from file_2.py to file_1.py the script runs smoothly in parallel. However, when I try to import the functions in combination with using ProcessPoolExecutor, I get the following error:

ModuleNotFoundError: No module named 'file_1'.

I would be great if anyone knows how to fix this! Thanks in advance :)

Here is a simplified version of my files:

# file_1.py

def addition(a,b):
    return a+b

# file_2.py

import functools
import numpy as np
import concurrent.futures
from file_1 import addition

if __name__ == '__main__':
    lst = np.arange(10)
    cons = 100
    
    partial = functools.partial(addition, b=cons)
    with concurrent.futures.ProcessPoolExecutor() as exc: 
        res = list(exc.map(partial, lst))
iClemens
  • 11
  • 1
  • maybe you start script in different folder and then it use different folder as Current Working Directory - check `os.getcwd()` - and then `import` search file in different folder. You may have to append foder with scripts to `sys.path` before `import` and it will know where to search. in `file_2` you may try to use `os.path.dirname(os.path.abspath(__file__))` to get full path to folder with `file_2.py` and then you can append it to `sys.path` – furas Apr 30 '21 at 19:54

0 Answers0