libraries which exist in a binary's elf RUNPATH are not being used?

Question

I have custom built gcc-4.7.2 in my environment. The system gcc is gcc-4.3.4.

I have patched the RUNPATH for all my custom gcc's binaries and shared libraries using patchelf --set-rpath

However, when I run ldd on my 4.7.2 cc1 it picks up the system libstdc++ instead of the one pointed to by the RUNPATH:

$ ldd /sdk/x86_64/2.11.1/gcc-4.7.2/libexec/gcc/x86_64-suse-linux/4.7.2/cc1
        libcloog-isl.so.1 => /sdk/x86_64/2.11.1/gcc-4.7.2/lib/libcloog-isl.so.1 (0x00007f072dce8000)
        ...
        libc.so.6 => /lib64/libc.so.6 (0x00007f072bfe0000)
   -->  libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f072bcd5000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f072babe000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f072df0d000)

As can be seen the RUNPATH specifies the gcc-4.7.2 library locations:

$ readelf -a /sdk/x86_64/2.11.1/gcc-4.7.2/libexec/gcc/x86_64-suse-linux/4.7.2/cc1 | grep PATH
 0x000000000000001d (RUNPATH)            Library runpath: \
    [/sdk/x86_64/2.11.1/gcc-4.7.2/lib64: \
     /sdk/x86_64/2.11.1/gcc-4.7.2/lib: \
     /sdk/x86_64/2.11.1/gcc-4.7.2/libexec/gcc/x86_64-suse-linux/lib64: \
     /sdk/x86_64/2.11.1/gcc-4.7.2/lib/gcc/x86_64-suse-linux/4.7.2: \
     /hostname/sig/lib64: \
     /hostname/sig/lib]

I know that libstdc++.so.6 exists in the first entry in the RUNPATH:

$ ls -l /sdk/x86_64/2.11.1/gcc-4.7.2/lib64/libstdc++.so*
lrwxrwxrwx .../sdk/x86_64/2.11.1/gcc-4.7.2/lib64/libstdc++.so -> libstdc++.so.6.0.17
lrwxrwxrwx .../sdk/x86_64/2.11.1/gcc-4.7.2/lib64/libstdc++.so.6 -> libstdc++.so.6.0.17
-rwxr-x--- .../sdk/x86_64/2.11.1/gcc-4.7.2/lib64/libstdc++.so.6.0.17
-rwxr-x--- .../sdk/x86_64/2.11.1/gcc-4.7.2/lib64/libstdc++.so.6.0.17-gdb.py

I don't have an LD_LIBRARY_PATH set in my environment:

$ echo $LD_LIBRARY_PATH

$

How come it doesn't pick up the library found in RUNPATH?
How can I force it to use the gcc-4.7.2 libraries?

score 6 · Accepted Answer · answered Jun 18 '13 at 00:57

The problem is that one of the prerequisites (libppl.so) also imports libstdc++. That prerequisite was built using the system gcc, and therefore finds /usr/lib64/libstdc++.so.6

$ ldd /sdk/x86_64/2.11.1/gcc-4.7.2/lib/libppl.so
        linux-vdso.so.1 =>  (0x00007fffd10db000)
        libgmpxx.so.4 => /sdk/x86_64/2.11.1/gcc-4.7.2/lib/libgmpxx.so.4 (0x00007f4716f92000)
        libgmp.so.10 => /sdk/x86_64/2.11.1/gcc-4.7.2/lib/libgmp.so.10 (0x00007f4716d26000)
    --> libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f4716a25000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f47167a0000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f4716441000)
        libgcc_s.so.1 => /usr/lib64/libgcc_s.so.1 (0x00007f471622c000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f47174b4000)

Once a library has been located by the dynamic linker once, it will no longer be searched for; that location will be used for any subsequent requirements.

I resolved this by rebuilding the prerequisites with the new gcc.

$ ldd /sdk/x86_64/2.11.1/gcc-4.7.2/lib/libppl.so
        linux-vdso.so.1 =>  (0x00007fffd10db000)
        libgmpxx.so.4 => /sdk/x86_64/2.11.1/gcc-4.7.2/lib/libgmpxx.so.4 (0x00007f4716f92000)
        libgmp.so.10 => /sdk/x86_64/2.11.1/gcc-4.7.2/lib/libgmp.so.10 (0x00007f4716d26000)
    --> libstdc++.so.6 => /sdk/x86_64/2.11.1/gcc-4.7.2/lib/../lib64/libstdc++.so.6 (0x00007f4716a25000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f47167a0000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f4716441000)
        libgcc_s.so.1 => /sdk/x86_64/2.11.1/gcc-4.7.2/lib/../lib64/libgcc_s.so.1 (0x00007f471622c000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f47174b4000)

I'm thinking the final step is to now rebuild gcc with the newly build prerequisites.

build prerequisites with system gcc
build new gcc
rebuild prerequisites with new gcc
rebuild gcc with rebuilt prerequisites

Whether the final step is necessary I'm not sure.

Hi. But could you, in principle, patch `libppl.so` with `patchelf` to change its' `RUNPATH` instead of rebuilding it with custom `gcc`? Or does `gcc` do some additional inexplicit magic, which works around `RUNPATH`? — xealits, Nov 01 '16 at 21:11

score 2 · Answer 2 · answered Jun 17 '13 at 08:53

2

You need to set the LD_LIBRARY_PATH to point to the desired libstdc++. RUNPATH is evaluated after LD_LIBRARY_PATH.

Quoting from RPATH issue:

The dynamic linker will look for a matching library in the following locations, in this order, which can be changed (see the footnotes below): 
1. the DT_RPATH dynamic section attribute of the library causing the lookup 
2. the DT_RPATH dynamic section attribute of the executable 
3. the LD_LIBRARY_PATH environment variable, unless the executable is setuid/setgid. 
4. the DT_RUNPATH dynamic section attribute of the executable 
5. /etc/ld.so.cache 
6. base library directories (/lib and /usr/lib)

answered Jun 17 '13 at 08:53

devnull

118,548
33
236
227

5

I don't want to use LD_LIBRARY_PATH. I think forcing users to change their environment is the wrong approach, and brittle. So in this situation there is no LD_LIBRARY_PATH, so RUNPATH should be used next right? If this is the case then how come it doesn't work? – Steve Lorimer Jun 17 '13 at 21:28
From the article you quoted: LD_LIBRARY_PATH can't be used because it has its own problems: it'll be inherited by all processes generated by the parent and is therefore also discouraged for distribution-wide use for its possible side-effects. – Steve Lorimer Jun 17 '13 at 22:32
2

@SteveLorimer It doesn't work because the search order in this answer isn't correct. Item 4 should be "the DT_RUNPATH dynamic section attribute of the library causing the lookup". In the context of the original question, libppl's lookup of libstdc++ does NOT use the RUNPATH from the executable. – TrentP Mar 14 '19 at 11:30

libraries which exist in a binary's elf RUNPATH are not being used?

2 Answers2