I would like to run a calculation on an IPython cluster that uses a custom numba-fied function to run quickly. I try to synchronize the imports using the sync_imports method of the dview object. The problem is that the engines don't seem to load numba correctly, while numpy get imported just fine... Here's a short "minimal" code that points to the problem:
from IPython.parallel import Client
rc = Client()
dview = rc[:]
print 'before import'
with dview.sync_imports():
import numpy
import numba
print 'no deadlock'
When I run this code there is a sort of deadlock happening after the numpy import, and after a while I get this error:
before import
importing numpy on engine(s)
importing numba on engine(s)
Traceback (most recent call last):
File "./ipcltest.py", line 12, in
import numba
File "/c5/shared/Python/2.7.4-shared/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
File "/c5/shared/Python/2.7.4-shared/lib/python2.7/site-packages/IPython/parallel/client/view.py", line 496, in sync_imports
r.get()
File "/c5/shared/Python/2.7.4-shared/lib/python2.7/site-packages/IPython/parallel/client/asyncresult.py", line 119, in get
self.wait(timeout)
File "/c5/shared/Python/2.7.4-shared/lib/python2.7/site-packages/IPython/parallel/client/asyncresult.py", line 142, in wait
self._ready = self._client.wait(self.msg_ids, timeout)
File "/c5/shared/Python/2.7.4-shared/lib/python2.7/site-packages/IPython/parallel/client/client.py", line 1063, in wait
self.spin()
File "/c5/shared/Python/2.7.4-shared/lib/python2.7/site-packages/IPython/parallel/client/client.py", line 1013, in spin
self._flush_results(self._mux_socket)
File "/c5/shared/Python/2.7.4-shared/lib/python2.7/site-packages/IPython/parallel/client/client.py", line 821, in _flush_results
raise Exception("Unhandled message type: %s" % msg_type)
Exception: Unhandled message type: apply_request
./ipcltest.py 1.20s user 0.40s system 0% cpu 25:29.11 total
Does anyone have any idea about what causes this behaviour ?
