How can the exact same binary library on distro X be linked to less libaries than on distro Y?

Question

I've got a 3rd party binary library for connecting to a server via SSL that works fine on Cent OS 4 32-bit, but on Debian Lenny 32-bit I get an SSL initialization error when trying to process a transaction.

When I execute ldd on the library on Debian, 5 links are missing that ldd on Cent OS has:

libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2
libkrb5.so.3 => /usr/lib/libkrb5.so.3
libcom_err.so.2 => /lib/libcom_err.so.2
libk5crypto.so.3 => /usr/lib/libk5crypto.so.3
libresolv.so.2 => /lib/libresolv.so.2

I suspect my problem lies here. All those libraries are installed on the Debian system so I'm perplexed that the 3rd party binary does not see them.

I've done an md5sum on the 3rd party binary on each system, & they are exactly the same.

Here is the complete ldd listing from Cent OS:

[root@localhost ~]# ldd /usr/lib/libwebpayclient.so
        libssl.so.4 => /lib/libssl.so.4 (0x0026a000)
        libcrypto.so.4 => /lib/libcrypto.so.4 (0x00c41000)
        libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0x00544000)
        libm.so.6 => /lib/tls/libm.so.6 (0x0093e000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00da4000)
        libc.so.6 => /lib/tls/libc.so.6 (0x0066e000)
        libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0x008dd000)
        libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x00394000)
        libcom_err.so.2 => /lib/libcom_err.so.2 (0x00111000)
        libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x00114000)
        libresolv.so.2 => /lib/libresolv.so.2 (0x007da000)
        libdl.so.2 => /lib/libdl.so.2 (0x00135000)
        libz.so.1 => /usr/lib/libz.so.1 (0x004d1000)
        /lib/ld-linux.so.2 (0x008b5000)

Note that I had to install the package compat-libstdc++-33.i386 to resolve libstdc++.so.5

And here is the complete ldd listing from Debian:

localhost:~# ldd /usr/lib/libwebpayclient.so
    linux-gate.so.1 =>  (0xb7fcb000)
    libssl.so.4 => not found
    libcrypto.so.4 => not found
    libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0xb7ee5000)
    libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7ebf000)
    libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7eb2000)
    libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7d57000)
    /lib/ld-linux.so.2 (0xb7fcc000)

Note that I had to install the package libstdc++5 to resolve libstdc++.so.5.

Using ln -s to fix the 2 "not found" links I get:

localhost:~# ldd /usr/lib/libwebpayclient.so
        linux-gate.so.1 =>  (0xb7eff000)
        libssl.so.4 => /usr/lib/libssl.so.4 (0xb7e8e000)
        libcrypto.so.4 => /usr/lib/libcrypto.so.4 (0xb7d31000)
        libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0xb7c76000)
        libm.so.6 => /lib/i686/cmov/libm.so.6 (0xb7c50000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7c43000)
        libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb7ae8000)
        libdl.so.2 => /lib/i686/cmov/libdl.so.2 (0xb7ae4000)
        libz.so.1 => /usr/lib/libz.so.1 (0xb7acf000)
        /lib/ld-linux.so.2 (0xb7f00000)

Interestingly, libz.so.1 appears. So there is the clue.

The version of SSL on Cent OS is 0.9.7a where it's 0.9.8 on Debian. I bet it's linked to less libraries...

womble · Answer 1 · 2009-08-10T02:59:02.417

Are those ldd runs from identical (md5sums the same) copies of the same library? If libraries are missing on one system or another, ldd should report "not found", it shouldn't just fail to display anything.

The initial list of libraries that a shared object (be it library or binary) requires is hard-coded, and short of recompilation or mangling, the list will never change. One thing that can upset the apple cart is that ldd walks the list of library dependencies and displays info on those libraries too, so you can legitimately have different lists of libraries on different systems because the libraries that are pointed to point to have different sets of included libraries. Confused yet?

Given the massive difference between the two lists, though, and the fact that you've got libraries in there I've never seen (I don't seem to have libssl.so.4 on my CentOS systems), I'm going to stick with "undisclosed information" as being the source of your problems -- something like installing a pile of extra libraries with crazy linkage info somewhere.

Also, your question isn't clear as to whether the first paste of ldd output is complete, or a "diff". Please include the full output of ldd from both systems, with any hand-made symlinks removed (and also include the list of symlinks you had to create, with their source and target locations).

I've updated my question, but yes they are the same binary. I've seen the "not found" message & had to fix some of those with sym links. Perhaps a newer version of a related lib has the function of the "missing" lib, hence the "missing" linked lib? Strange to me, & no-one at my work has heard of this before. — Glen Richards, Aug 10 '09 at 02:08

Glen Richards · Accepted Answer · 2009-08-10T22:54:26.563

The problem was that the version of SSL on Debian (libssl.so.0.9.8) no longer linked to the files that libssl.so.0.9.7a on Cent OS did. By copying the CentOS version on to the Debian box & fixing the /usr/lib/libssl.so.4 link, the problem was solved. This is not ideal for security purposes, because if Cent OS updates that file, I'll have to remember to copy it across. I've subscribed to Cent OS 4 security advisories for this purpose. Hopefully the 3rd party releases an updated library soon.

How can the exact same binary library on distro X be linked to less libaries than on distro Y?

2 Answers2