12

I am trying to access a shared library with OpenMPI via python, but for some reason I get the following error message:

[Geo00433:01196] mca: base: component_find: unable to open /usr/li/openmpi/lib/openmpi/mca_paffinity_hwloc: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[Geo00433:01196] mca: base: component_find: unable to open /usr/lib/openmpi/lib/openmpi/mca_carto_auto_detect: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[Geo00433:01196] mca: base: component_find: unable to open /usr/lib/openmpi/lib/openmpi/mca_carto_file: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[Geo00433:01196] mca: base: component_find: unable to open /usr/lib/openmpi/lib/openmpi/mca_shmem_mmap: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[Geo00433:01196] mca: base: component_find: unable to open /usr/lib/openmpi/lib/openmpi/mca_shmem_posix: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[Geo00433:01196] mca: base: component_find: unable to open /usr/lib/openmpi/lib/openmpi/mca_shmem_sysv: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
-------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here is some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
    --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
[Geo00433:01196] [[INVALID],INVALID] ORTE_ERROR_LOG: Error in file runtime/orte_init.c at line 79
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here is some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: orte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL: your MPI job will now abort
[Geo00433:1196] Local abort before MPI_INIT completed successfully; not able to aggregate error messages, and not able to guarantee that all other processes were killed!

Any clue what's the reason? I checked many webpages already, but somehow couldn't find a solution for my problem yet.

I have Ubuntu 15.10 and mpich as well as open-mpi installed.

Thanks a lot guys!

Jannis
  • 173
  • 2
  • 6
  • Please provide your code. – Jan Mar 22 '16 at 14:35
  • 3
    Your program was probably compiled against `libmpi.so` from MPICH but then finds the Open MPI version during runtime instead. Having two MPI implementations at the same time often leads to such problems. – Hristo Iliev Mar 22 '16 at 15:57
  • Indeed, it was related to finding the correct OpenMPI version. Thanks @HristoIliev – Jannis May 23 '16 at 08:08
  • Look at here http://stackoverflow.com/questions/26901663/error-when-running-openmpi-based-library – AdityaG Sep 27 '16 at 14:59

5 Answers5

5

I had the same problem (or very similar with slightly different error message) on Ubuntu 16.04, even with only Open MPI installed. From what I can tell there is a problem with how the mpi4py package from Ubuntu was built, but am not sure what exactly it is.

Reproduction: Since the question doesn't make it entirely clear how the error message was produced (I don't have the reputation to edit it), here's how I got it. First, install Ubuntu's mpi4py package and then enter the python environment:

$ sudo apt-get install mpi
$ python

Inside python, try the following:

>>> from mpi4py import MPI

You should then get an error message like the OP had.

Solution: Here is how I got it working. First uninstall Ubuntu's package:

$ sudo apt-get remove mpi4py

Then install the Open MPI headers (the next step involves building mpi4py) and pip:

$ sudo apt-get install libopenmpi-dev python-pip

Finally install mpi4py:

$ sudo pip install mpi4py

If you try the python command above, it should now work fine.

fltfan
  • 51
  • 3
  • Seems to be a good workaround for now; I'm linking the known (currently open) issue https://bugs.launchpad.net/ubuntu/+source/mpi4py/+bug/1583432 – bbarker Feb 08 '17 at 19:15
0

The error message was indeed related to the different .so-files as Hristo Iliev indicated. When compiling the program I use, the compiler found the "wrong" OpenMPI on my Linux machine, i.e. by explicitly specifying with OpenMPI to use, the problem was solved.

Thanks for your help guys!

Jannis
  • 173
  • 2
  • 6
0

I also got the similar error when I use the python interface for MPI wrapped by myself with SWIG. This error is maybe related the different versions of MPI implementation on the same machine(e.g the OpenMPI and MPICH both on your computer) as mentioned above.

I solved this problem by compiling and installing a new version of MPICH . then changed the environment variables in .bashrc and compile my own program with the new mpicxx or mpicc. The error would disappear.

PytLab
  • 539
  • 6
  • 12
0

I got a similar error when trying to use mpi4py on Ubuntu 16.04 LTS. The error, in my case, was related to the fact that the mpicc wrapper was not in my search path.

What I did to solve the problem was the following

  • Uninstall your current mpi4py with

$ sudo pip uninstall mpi4py

  • find the path to your mpicc with

$ which mpicc

$ sudo env MPICC=/path/to/mpicc pip install mpi4py

After that the error messages disappeared and I was able to run MPI with python

ste
  • 1
0

try uninstalling your mpi4py at first

>>> pip uninstall mpi4py

then

>>> conda install mpi4py

source: https://nyu-cds.github.io/python-mpi/setup/

Wang Wei
  • 39
  • 3