python multiprocessing module cause infinite loop

Question

I am trying to use pool from multiprocessing to speed up my read/write function. However, when I try to use pyinstaller and run it with an exe. The result is a new process that spawns a new process that spawns a new process ad infinitum until I log out and the OS forcefully terminates all my user processes. I have no idea what is happening

There is no problem when running in the pycharm or even running the code with running the .py.

The version is python 3.6.3.

Platform: Windows 7

The code will be like:

if __name__ == "__main__":
    pool = Pool(2)
    pool.map(readwritevalue, [file for file in gettxtpath()])
    pool.close()
    pool.join()
    GetFormated()  ##Get the Final output of .csv
    Getxlsx()  ##Get the Report.xls
    GetDocx()  ##Get the docx format
    t1 = time.time()
    print("Parallel time{t}s".format(t=time.time() - t1))

Would you please shed some lights on this? I searched the Google and some answers are to put the multiprocessing module under the "__name__=="__main__"", However, I already did that. So what else will cause that infinite explosion of processes?

Thank you very much.

Have you checked [this](https://stackoverflow.com/questions/32672596/pyinstaller-loads-script-multiple-times) out? Apparently pyinstaller has some issues w/ multiprocessing modules. — Milan Velebit, Jul 20 '18 at 12:09
I'm not going to take credit for the resolution, but please post the answer with your modified code below just in case someone else stumbles upon it, glad it helped! — Milan Velebit, Jul 23 '18 at 07:18
@MilanVelebit I have posted the answer. Thanks again for your great help! — Winnie-c, Jul 23 '18 at 12:29

score 1 · Answer 1 · answered Jul 23 '18 at 12:28

Thanks MilanVelebit for helping to find out the answer.

I post my answer here. You just need to add one line when you import multiprocessing and use pyinstaller.

from multiprocessing import Pool,freeze_support

if __name__ == "__main__":

    ##Add support for when a program which uses multiprocessing has been frozen to produce a Windows executable. (Has been tested with py2exe, PyInstaller and cx_Freeze.)
    #One needs to call this function straight after the '__main__' line of the main module.

    freeze_support()
    t1 = time.time()
    pool = Pool(2)
    pool.map(readwritevalue, [file for file in gettxtpath()])
    pool.close()
    pool.join()
    GetFormated()  ##Get the Final output of Xunjian-Report.csv
    Getxlsx()  ##Get the Xunjian-Report.xls
    GetDocx()  ##Get the docx format
    print("Cost time {t}".format(t=time.time() - t1))

python multiprocessing module cause infinite loop

1 Answers1