How can I resolve the "import multiarray" error in parallel python?

Question

I am trying to run a python 2.7 script which uses parallel python (version 1.6.1) to excute a function which uses numpy arrays (numpy version 1.6.1) on a Ubuntu Voyager (Ubuntu 12.04 derivative) system. It gives me the following error message (actually, it's still longer, repeated 12 times I guess):

Starting pp with 12 workers
* An error has occured during the module import
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/ppworker.py", line 49, in preprocess
    exec module
  File "<string>", line 1, in <module>
  File "/usr/share/pyshared/numpy/__init__.py", line 137, in <module>
    import add_newdocs
  File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
    from numpy.lib import add_newdoc
  File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
    from type_check import *
  File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
    import numpy.core.numeric as _nx
  File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
    import multiarray
ImportError: No module named multiarray
A fatal error has occured during the function execution
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.7/ppworker.py", line 86, in run
    __args = pickle.loads(__sargs)
  File "/usr/share/pyshared/numpy/__init__.py", line 137, in <module>
    import add_newdocs
  File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
    from numpy.lib import add_newdoc
  File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
    from type_check import *
  File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
    import numpy.core.numeric as _nx
  File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
    import multiarray
ImportError: No module named multiarray
Exception in thread run_local:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/pymodules/python2.7/pp.py", line 719, in _run_local
    job.finalize(sresult)
UnboundLocalError: local variable 'sresult' referenced before assignment

Exception in thread run_local:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/pymodules/python2.7/pp.py", line 719, in _run_local
    job.finalize(sresult)
UnboundLocalError: local variable 'sresult' referenced before assignment

From other threads (e.g. importing NumPy in Parallel Python) I have seen that it's probably an issue with the communication of pp and numpy, so I have tried these:

updating pp and numpy
reinstalling numpy (I'm using the anaconda package)
running the code directly from the shell (not from spyder)
submitting multiarray (which otherwise imports just fine) directly to the workers with the job (and I'm sure I have the correct syntax ...)

but to no avail. Do you have any other suggestions? Is there a simple workaround other than, erm, "de-numpying" my function (which would be sad, it's a matrix multiplication)? My colleague who uses a more recent version of the same operating system, but otherwise the same python setup, does not seem to have this problem, but since time is an issue updating my system or hijacking his slower computer are both not the best options.

If necessary, feel free to ask for more details & thank you in advance.

How can I resolve the "import multiarray" error in parallel python?

0 Answers0