1

I have Python code that works on Windows, however when running on Linux it just hangs. I'm using JPype, so I suspect that there might be some issue with multiple shared processes trying to use the same pipe to access Java (the different processes are created but hang at the JPype line). Is there any way to enforce spawning in Pathos to copy the Windows implementation? (e.g. set_start_method, or get_context in the regular multiprocessing library?)

Thanks.

morganics
  • 1,209
  • 13
  • 27

1 Answers1

5

To answer my own question (and it's a bit nasty) but after digging through the code you can:

import multiprocess.context as ctx
ctx._force_start_method('spawn')

Which happily solves the issue I'm having with JPype hanging. The difference between Linux and Windows is that as Windows spawns a new process, a new JVM is started (jpype.startJVM()), whereas a forked process has to use the same one (so I'm guessing that there are multiple processes trying to use the same pipe to Java). set_start_method doesn't seem to have been implemented, as far as I can tell.

morganics
  • 1,209
  • 13
  • 27
  • Hmm… that's a good solution. I was thinking it might be possible to use the `ForkingPickler` in `multiprocess`, which gives windows pickling behavior (i.e. zero shared objects)... but I think you've probably hit on the right way to do it. – Mike McKerns Nov 21 '16 at 22:28