4

The code below works perfectly on Unix but generates a multiprocessing.TimeoutError on Windows 7 (both OS use python 2.7).

Any idea why? Thanks.

from multiprocessing import Pool

def increment(x):
    return x + 1

def decrement(x):
    return x - 1

pool = Pool(processes=2)
res1 = pool.map_async(increment, range(10))
res2 = pool.map_async(decrement, range(10))

print res1.get(timeout=1)
print res2.get(timeout=1)
David M.
  • 4,518
  • 2
  • 20
  • 25

1 Answers1

1

You need to put your actual program logic in side a if __name__ == '__main__': block.

On Unixy systems, Python forks, producing multiple processes to work from. Windows doesn't have fork. Python has to launch a new interpreter and re-import all your modules instead. This means that each subprocess will reimport your main module. For the code you've written reimporting the module will cause each newly launched processes to launch processes of its own.

See: http://docs.python.org/library/multiprocessing.html#windows

EDIT this works for me:

from multiprocessing import Pool

def increment(x):
    return x + 1

def decrement(x):
    return x - 1

if __name__ == '__main__':
    pool = Pool(processes=2)
    res1 = pool.map_async(increment, range(10))
    res2 = pool.map_async(decrement, range(10))

    print res1.get(timeout=1)
    print res2.get(timeout=1)
Winston Ewert
  • 44,070
  • 10
  • 68
  • 83
  • Thanks for your answer but the snippet above is actually part of a larger script which already includes `if __name__ == '__main__'`. Anyway, I've tried to add `if __name__ == '__main__'` to the snippet only and it still does not work. Even weirder, the `print 'hello'`example on the Python doc page you mention doesn't work either (it doesn't print anything). – David M. Sep 17 '12 at 19:03
  • @user1453508, I've added my adjustment of your code that works for me. Is it the same thing you did? Try adding `p.join()` to the example from the python docs. – Winston Ewert Sep 17 '12 at 21:04
  • Yes, if I copy/paste your adjusted code, it doesn't work (same Timeout Error); if I add p.join() to the example from the python docs it doesn't work either. Which version of python and Windows are you using? – David M. Sep 17 '12 at 22:35
  • @user1453508, I'm on a different computer now but I believe that it was Windows 7 with Python 2.7. Can you copy/paste your exact error? – Winston Ewert Sep 17 '12 at 22:56
  • Traceback (most recent call last): File "my_test.py", line 14, in print res1.get(timeout=1) File "C:\Python27\lib\multiprocessing\pool.py", line 518, in get raise TimeoutError multiprocessing.TimeoutError – David M. Sep 18 '12 at 08:20
  • I have found that if I replace `p.start()` with `p.run()` in the python doc example, it works, but I have no idea why. Also this doesn't seem to explain why the map_async snippet does not work. – David M. Sep 18 '12 at 09:22
  • @user1453508, calling `p.run()` will run the code in the current process, not the external process. It sounds to me like something is messed up in the python installation. I'd uninstall and reinstall python and see if that helps. – Winston Ewert Sep 18 '12 at 12:35
  • I have reinstalled Python and everything works fine now. Thanks for your help. – David M. Sep 19 '12 at 08:34