RuntimeError on windows trying python multiprocessing

Question

I'm going to dump the error code I got while try a python script :

Preprocess validation data upfront

Using gpu device 0: Tesla K20c

Traceback (most recent call last):

File "<string>", line 1, in <module>

File "C:\SciSoft\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\multiprocessing\forking.py", line 380, in main prepare(preparation_data)

File "C:\SciSoft\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\multiprocessing\forking.py", line 495, in prepare '__parents_main__', file, path_name, etc

File "C:\Users\Administrator\Desktop\Galaxy Data\kaggle-galaxies-master\kaggle-galaxies-master\try_convnet_cc_multirotflip_3x69r45_maxout2048_extradense.py", line 133, in <module>

for data, length in create_valid_gen():

File "load_data.py", line 572, in buffered_gen_mp process.start()

`File "C:\SciSoft\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\multiprocessing\process.py", line 130, in start self._popen = Popen(self)

File "C:\SciSoft\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\multiprocessing\forking.py", line 258, in init cmd = get_command_line() + [rhandle]

File "C:\SciSoft\WinPython-64bit-2.7.6.4\python-2.7.6.amd64\lib\multiprocessing\forking.py", line 358, in get_command_line`

is not going to be frozen to produce a Windows executable.''')

RuntimeError: Attempt to start a new process before the current process has finished its bootstrapping phase.

        This probably means that you are on Windows and you have
        forgotten to use the proper idiom in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce a Windows executable.

As I understand I have to insert a line

if __name__ == '__main__':

Some where to get it to work

Can anyone tell me in which File I should insert it ? I have included the affected files list in the initial error logs

The affected file :

https://github.com/benanne/kaggle-galaxies/blob/master/try_convnet_cc_multirotflip_3x69r45_maxout2048_extradense.py

Lines 131-134

and

https://github.com/benanne/kaggle-galaxies/blob/master/load_data.py

line 572

Could you show us your code? – laike9m Mar 03 '15 at 11:33 — laike9m, Mar 03 '15 at 11:33
Yes provided links to files – Thalish Sajeed Mar 03 '15 at 12:31 — Thalish Sajeed, Mar 03 '15 at 12:31

score 0 · Accepted Answer · answered Mar 03 '15 at 14:57

Python documentation is quite clear in this case.
The important part is Safe importing of main module.

Your try_convnet_cc_multirotflip_3x69r45_maxout2048_extradense.py script is doing lots of things on module level. Without reading it in details I can already say that you should wrap the workflow with a function and use it like this:

if __name__ == '__main__':
    freeze_support() # Optional under circumstances described in docs
    your_workflow_function()

Besides the problem you have, it's a good habit not to surprise possible user of your script with side effects, if the user just wants to import it and reuse some of it's functionality.
So don't put your code on module level. It's ok to have constants on module level but the workflow should be in functions and classes.
If Python module is intended to be used as a script (like in your case), you simply put the if __name__ == '__main__' in the very end of this module, calling your_workflow_function() only if the module is the entry point for the interpreter - so called main module.

RuntimeError on windows trying python multiprocessing

1 Answers1