Undefined references when mixing Intel C++ and Fortran in OpenMPI

Question

I have a compilation problem using Open MPI 1.8.4 and Intel Compiler v15.2. This is a large code that uses Fortran and C++. The code was previously compiled using Open MPI 1.6 And the issue was not there.

Here is the content of the make file:

ifdef TAUDEBUG
    FC=tau_f90.sh
    COMP=tau_f90.sh
    CXX=tau_cxx.sh
    CXXFLAGS=
    CXXOPT=
    LINKER=tau_cxx.sh
else
    FC=mpif90
    COMP=mpif90
    CXX=mpic++
    CXXFLAGS=
    CXXOPT=
    LINKER=$(CXX)
endif


CXXLINKFLAGS += -I/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,-rpath -Wl,/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,--enable-new-dtags -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

FORTFLAGS= -DFORTRAN_UNDERSCORE -DFORTRAN_LOWERCASE

The issue I am having is that C++ code is using some of the Fortran routines and mpic++ needs to be linked with Fortran libraries.

With Open MPI 1.6 it was done with:

CXXLINKFLAGS += -L/usr/mpi/intel/openmpi-1.6/lib64 -lmpi_f90

And it worked.

Starting with OpenMPI v.1.7 mpif90 was deprecated. and libmpi_f90 no longer exists. All Fortran related routines are handled by libmpi_usempif08* and libmpi_mpifh*.

And that is where I am struggling.

I tried

CXXLINKFLAGS += -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_mpifh -lmpi -lmpi_usempi_ignore_tkr

And the Fortran portion of the code compiles fine but when it gets to mpic++ I start getting a lot of "undefined reference to" errors:

mpic++  rflump.o main.o -o ../rflump -O .././build_lib/libflu.a      -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_mpifh -lmpi -lmpi_usempi_ignore_tkr 

rflump.f90:(.text+0x56): undefined reference to `for_cpystr'
.././build_lib/libflu.a(RFLU_EndFlowSolver.o): In function `rflu_endflowsolver_':
RFLU_EndFlowSolver.f90:(.text+0x2a): undefined reference to `for_cpystr'
.././build_lib/libflu.a(RFLU_FlowSolver.o): In function `rflu_flowsolver_':
RFLU_FlowSolver.f90:(.text+0x2d): undefined reference to `for_cpystr'
.././build_lib/libflu.a(RFLU_InitFlowSolver.o): In function `rflu_initflowsolver_':
RFLU_InitFlowSolver.f90:(.text+0x4e): undefined reference to `for_cpystr'
RFLU_InitFlowSolver.f90:(.text+0x1b3): undefined reference to `for_inquire'
RFLU_InitFlowSolver.f90:(.text+0x1c59): undefined reference to `for_cpstr'
RFLU_InitFlowSolver.f90:(.text+0x2cc4): undefined reference to `for_write_seq_lis'
.././build_lib/libflu.a(RFLU_OpenConverFile.o): In function `rflu_openconverfile_':
RFLU_OpenConverFile.f90:(.text+0x2d): undefined reference to `for_cpystr'
RFLU_OpenConverFile.f90:(.text+0x1ca): undefined reference to `for_trim'
RFLU_OpenConverFile.f90:(.text+0x20a): undefined reference to `for_concat'
.././build_lib/libflu.a(RFLU_OpenPMFile.o): In function `rflu_openpmfile_':
RFLU_OpenPMFile.f90:(.text+0x2d): undefined reference to `for_cpystr'
RFLU_OpenPMFile.f90:(.text+0x1ca): undefined reference to `for_trim'
RFLU_OpenPMFile.f90:(.text+0x20a): undefined reference to `for_concat'
.././build_lib/libflu.a(RFLU_OpenStatsFileOLES.o): In function `rflu_openstatsfileoles_':
RFLU_OpenStatsFileOLES.f90:(.text+0x2d): undefined reference to `for_cpystr'
RFLU_OpenStatsFileOLES.f90:(.text+0x1c0): undefined reference to `for_trim'
RFLU_OpenStatsFileOLES.f90:(.text+0x200): undefined reference to `for_concat'
.././build_lib/libflu.a(RFLU_OpenTotalMassFile.o): In function `rflu_opentotalmassfile_':
RFLU_OpenTotalMassFile.f90:(.text+0x2d): undefined reference to `for_cpystr'
RFLU_OpenTotalMassFile.f90:(.text+0x1ca): undefined reference to `for_trim'
RFLU_OpenTotalMassFile.f90:(.text+0x20a): undefined reference to `for_concat'
.././build_lib/libflu.a(RFLU_PrintHeader.o): In function `rflu_printheader_':
RFLU_PrintHeader.f90:(.text+0x26): undefined reference to `for_cpystr'
RFLU_PrintHeader.f90:(.text+0x6e): undefined reference to `for_cpystr'
RFLU_PrintHeader.f90:(.text+0x7d): undefined reference to `for_len_trim'
RFLU_PrintHeader.f90:(.text+0xc2): undefined reference to `for_cpystr'
RFLU_PrintHeader.f90:(.text+0x324): undefined reference to `for_trim'
RFLU_PrintHeader.f90:(.text+0x622): undefined reference to `for_trim'
RFLU_PrintHeader.f90:(.text+0x77a): undefined reference to `for_trim'

I know that OpenMPI is using right compiler:

$ mpifort -V
Intel(R) Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 15.0.2.164 Build 20150121
Copyright (C) 1985-2015 Intel Corporation.  All rights reserved.

$ mpic++ -V
Intel(R) C++ Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 15.0.2.164 Build 20150121
Copyright (C) 1985-2015 Intel Corporation.  All rights reserved.

Also according to OpenMPI documentation to link to Fortran I need to use:

$ mpifort --showme:link
-I/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,-rpath -Wl,/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,--enable-new-dtags -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

I copied the whole line and pasted it to Makefile:

CXXLINKFLAGS += -I/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,-rpath -Wl,/cm/shared/apps/openmpi/intel/1.8.4/lib -Wl,--enable-new-dtags -L/cm/shared/apps/openmpi/intel/1.8.4/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi

The result was identical to the previous output. It looks like that C++ compiler is trying to use Fortran but is not finding it.

What am I missing here?

Try running `objdump` or `nm` on the Fortran libraries you're linking against to get the list of symbols, then `grep for_cpystr`. What do you see? — John Zwinck, Apr 14 '16 at 00:38
The references to `for_trim` etc are just standard Fortran functions `trim` and the such. I'm pretty sure than you need to link with `mpifort` and not `mpicxx` to get these functions. — RussF, Apr 14 '16 at 02:11
I don't remember it by heart, but the Intel Fortran standard library is called something like `libifort.so`, therefore you should add `-lifort` to the linker flags. Another option is to use `mpifort -nofor_main ...` to link the executable. The `-nofor_main` should be there if the main function of your program is part of the C/C++ code and the Fortran one. — Hristo Iliev, Apr 14 '16 at 06:39
I think Hristo is right and this is not a OpenMPI issue. I'd expect the same issue when you link a serial non-MPI program this way. — Vladimir F Героям слава, Apr 14 '16 at 07:43
I just checked - the standard Fortran library in `ifort` is split into multiple DSOs. You should really be using `mpifort` to link the executable. — Hristo Iliev, Apr 14 '16 at 11:03

score 1 · Answer 1 · edited Apr 14 '16 at 11:30

1

Like Hristo suggested, I added -lifcore (that was the library I found in Intel compilers) to the CXXLINKFLAGS and I do not see the undefined reference errors any more. I did not realize that you can add icpc and ifort options to mpic++ and mpifort that way.

edited Apr 14 '16 at 11:30

Vladimir F Героям слава

57,977
4
76
119

answered Apr 14 '16 at 11:24

stas_s

71
1
4

Is is better to leave out the "chatty" stuff like Thank you! to comments or leave it out completely. This site should be a repository of problems and solutions without the need to read too much stuff around when your are searching for a solution. – Vladimir F Героям слава Apr 14 '16 at 11:31
As Hristo points out above, it might be better to link with `mpifort` and add the C++ libraries instead. – Vladimir F Героям слава Apr 14 '16 at 11:35

Undefined references when mixing Intel C++ and Fortran in OpenMPI

1 Answers1