2

I'm new to MPI, but I have been trying to use it on a cluster with OpenMPI. I'm having the following problem:

$ python -c "from mpi4py import MPI"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: libmpi.so.1: cannot open shared object file: No such file or directory

I've done some research and tried a few things including:

module load mpi/openmpi-x86_64

which doesn't seem to change anything. My LD_LIBRARY_PATH seems to be set correctly, but the desired "libmpi.so.1" does not exist. Instead, there is "libmpi.so.12":

$ echo $LD_LIBRARY_PATH 
/usr/lib64/openmpi/lib:/usr/local/matio/lib:/usr/lib64/mysql:/usr/local/knitro/lib:/usr/local/gurobi/linux64/lib:/usr/local/lib:
$ls /usr/lib64/openmpi/lib
... libmpi.so libmpi.so.12 libmpi.so.12.0.0 libmpi_usempi.so ...

I can't uninstall/re-install mpi4py because it's outside my virtual environment / I don't have the permissions to uninstall it on the general cluster. I've seen this: Error because file libmpi.so.1 missing, but not sure how I can create a symbolic link / don't think I have permission to modify the folder.

I'm somewhat out of ideas: Not sure if it's possible to have a separate install of mpi4py on my virtualenv, or whether I can specify the correct file to mpi4py temporarily?

Any suggestions would be greatly appreciated!

Update: I ran find /usr -name libmpi.so.1 2>/dev/null as suggested, which returned usr/lib64/compat-openmpi16/lib/libmpi.so.1. Using export LD_LIBRARY_PATH=$LD_LIBRARY_PATH"/usr/lib64/compat-openmpi16/lib/", python -c "from mpi4py import MPI" runs without any problems. However, if I try mpiexec -np 2 python -c 'from mpi4py import MPI', I get the following error:

[[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file ess_env_module.c at line 367
[[INVALID],INVALID]-[[28753,0],0] mca_oob_tcp_peer_try_connect: connect to 255.255.255.255:37376 failed: Network is unreachable (101)
Community
  • 1
  • 1
John Smith
  • 21
  • 1
  • 3
  • Ask the system administrators, which MPI implementation mpi4py is built against. `libmpi.so.12` is the _soname_ (shared object name) of the shared library that ships as part of Open MPI 1.10. The soname in earlier Open MPI versions is `libmpi.so.1`. – Hristo Iliev May 30 '16 at 18:13
  • Will do! I was hoping I could figure it out without the administrators, but seems unlikely. – John Smith May 30 '16 at 19:37
  • You could always run `find /usr -name libmpi.so.1 2>/dev/null` and then try to deduce the module name somehow from the path (if `find` finds anything) – Hristo Iliev May 30 '16 at 19:38
  • Thanks for all the help! I tried your suggestion, but I'm not sure if I should just use the path to change `LD_LIBRARY_PATH`. I put some updates in the question itself. – John Smith May 30 '16 at 21:22
  • 1
    For reference, something like `mpiexec -np 2 python -c "print 'Hello World'"` still runs as expected, so I believe it is still an issue with using mpi4py. – John Smith May 30 '16 at 21:23
  • You have to update both `PATH` and `LD_LIBRARY_PATH` to point to the location of that MPI version, otherwise you might end up with `mpiexec` from one version and shared libraries from a different one, which won't work. – Hristo Iliev May 31 '16 at 13:20
  • Got everything to work! Thanks so much for the help! – John Smith Jun 01 '16 at 20:42
  • Just for the record for those that do have permissions to create a link: "sudo ln -s libmpi.so.12.0.5 libmpi.so.1" in the library folder will probably do the trick. – pevogam Jan 11 '17 at 14:30

2 Answers2

1

Even adding to LD_LIBRARY_PATH did not work for me. I had to uninstall mpi4py, then install it manually with my mpicc path according to the instructions here - https://mpi4py.readthedocs.io/en/stable/install.html (using distutils section)

Sridhar Thiagarajan
  • 580
  • 1
  • 7
  • 20
-1

I solve this problem by

pip uninstall mpi4py
conda install mpi4py
Wendell Philips
  • 9
  • 4