0

I am running into errors finding the correct library

ERROR: ld.so: object '/ws/sarvi-sjc/cel8root/$LIB/libwisktrack.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.

My libraries are setup as follows

bash-4.4$ ls -alR /ws/sarvi-sjc/cel8root/
/ws/sarvi-sjc/cel8root/:
total 36
drwxr-xr-x.  6 sarvi eng  4096 Sep 23 22:46 .
drwxr-xr-x. 88 sarvi eng 16384 Sep 23 22:12 ..
drwxr-xr-x.  2 sarvi eng  4096 Sep 23 22:56 lib
drwxr-xr-x.  2 sarvi eng  4096 Sep 23 22:56 lib32
drwxr-xr-x.  2 sarvi eng  4096 Sep 23 22:56 lib64

/ws/sarvi-sjc/cel8root/lib:
total 29452
drwxr-xr-x. 2 sarvi eng     4096 Sep 23 22:56 .
drwxr-xr-x. 6 sarvi eng     4096 Sep 23 22:46 ..
-rwxrwxrwx. 1 sarvi eng 30024352 Sep 23 22:38 libwisktrack.so

/ws/sarvi-sjc/cel8root/lib32:
total 29428
drwxr-xr-x. 2 sarvi eng     4096 Sep 23 22:56 .
drwxr-xr-x. 6 sarvi eng     4096 Sep 23 22:46 ..
-rwxrwxrwx. 1 sarvi eng 30003020 Sep 23 22:56 libwisktrack.so

/ws/sarvi-sjc/cel8root/lib64:
total 28792
drwxr-xr-x. 2 sarvi eng     4096 Sep 23 22:56 .
drwxr-xr-x. 6 sarvi eng     4096 Sep 23 22:46 ..
-rwxrwxrwx. 1 sarvi eng 29349680 Sep 23 22:56 libwisktrack.so
bash-4.4$ 
bash-4.4$ file /ws/sarvi-sjc/cel8root/lib/libwisktrack.so 
/ws/sarvi-sjc/cel8root/lib/libwisktrack.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=c38570777cfc99005a4693f68adf1014b7d647fd, with debug_info, not stripped, too many notes (256)
bash-4.4$ file /ws/sarvi-sjc/cel8root/lib32/libwisktrack.so 
/ws/sarvi-sjc/cel8root/lib32/libwisktrack.so: ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically linked, BuildID[sha1]=be90026de8fad17a99582991f1983e321a17ea87, with debug_info, not stripped, too many notes (256)
bash-4.4$ file /ws/sarvi-sjc/cel8root/lib64/libwisktrack.so 
/ws/sarvi-sjc/cel8root/lib64/libwisktrack.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=223ca6c07b59b940355a3ca8299df8f68743c22a, with debug_info, not stripped, too many notes (256)
bash-4.4$

From what I can tell my library installation of 32 and 64 bit versions of my libwisktrack.so are installed right.

When i compile my own testprog32 and testprog64 they seem to load the respective libraries correctly

bash-4.4$ file tests/testprog32 
tests/testprog32: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=a9db0c61d648adf715a79c5cfb7b06f6a128b90e, not stripped
bash-4.4$ LD_PRELOAD='/ws/sarvi-sjc/cel8root/$LIB/libwisktrack.so' tests/testprog32
Constructor: 92729
testprog running......
Command: takes one argument. Which system call to run
Options: readlink, vprintf, printf, open, fopen, creat

bash-4.4$ file tests/testprog64
tests/testprog64: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=fbb14df720b2997c56995c98b2ad28734f554bb9, not stripped
bash-4.4$ LD_PRELOAD='/ws/sarvi-sjc/cel8root/$LIB/libwisktrack.so' tests/testprog64
Constructor: 92349
testprog running......
Command: takes one argument. Which system call to run
Options: readlink, vprintf, printf, open, fopen, creat

Yet this program has trouble find the right library and seems to pick up the wrong library

bash-4.4$ file /nobackup/sarvi/xrwisktest/binos/linkfarm/host/sdk/sysroots/x86_64-xesdk-linux/usr/bin/protoc
/nobackup/sarvi/xrwisktest/binos/linkfarm/host/sdk/sysroots/x86_64-xesdk-linux/usr/bin/protoc: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /nobackup/sarvi/xrwisktest/binos/linkfarm/host/sdk/sysroots/x86_64-xesdk-linux/lib/ld-linux-x86-64.so.2, BuildID[sha1]=f52e087673df429ad5849877b08d89b080e934c2, for GNU/Linux 2.6.32, stripped
bash-4.4$ 

bash-4.4$ LD_PRELOAD='/ws/sarvi-sjc/cel8root/$LIB/libwisktrack.so' /nobackup/sarvi/xrwisktest/binos/linkfarm/host/sdk/sysroots/x86_64-xesdk-linux/usr/bin/protoc
ERROR: ld.so: object '/ws/sarvi-sjc/cel8root/$LIB/libwisktrack.so' from LD_PRELOAD cannot be preloaded (wrong ELF class: ELFCLASS32): ignored.
Usage: /nobackup/sarvi/xrwisktest/binos/linkfarm/host/sdk/sysroots/x86_64-xesdk-linux/usr/bin/protoc [OPTION] PROTO_FILES

What am I missing here?

One difference I notice between the 64 bit testprog64 and the failing 64 bit program is

interpreter /lib64/ld-linux-x86-64.so.2
Vs 
interpreter /nobackup/sarvi/xrwisktest/binos/linkfarm/host/sdk/sysroots/x86_64-xesdk-linux/lib/ld-linux-x86-64.so.2

Not sure how that maters

Update: strace shows it opening the following file for a 64 bit program invocation, which doesnt make sense to me

open("/ws/sarvi-sjc/wisktrack/cel8root/lib/libwisktrack.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

Shouldn't this be opening one of the lib64 or lib/x86_64 or lib/x86_64-linux-gnu paths ?

Just to be clear, I don't want to use LD_LIBRRY_PATH, as I ran into use cases where some programs/scripts, explicitly disallow the setting of LD_LIBRARY_PATH, for example "python setup.py install" I found in a SDK.

So I looking to make this work without having to use LD_LIBRARY_PATH. "$LIB" to work consistently work is the only option i am aware of.

Mine is a RHEL8 development host. The specific program that is running into a problem extracted from an SDK compiled for the host to build other cross compiling programs as art of a very large build system.

Sarvi Shanmugham
  • 487
  • 4
  • 13

2 Answers2

1

$LIB documentation from ld.so man: https://man7.org/linux/man-pages/man8/ld.so.8.html

   $LIB (or equivalently ${LIB})
          This expands to lib or lib64 depending on the architecture
          (e.g., on x86-64, it expands to lib64 and on x86-32, it
          expands to lib).

This is not completely true and actual values can be different. E.g. you are running a nativesdk binary from OpenEmbedded SDK. As you've noticed the SDK uses its own ld.so. That ld.so is not configured for multilib support and it expects to have only one lib directory with 64-bit libraries. That's why it resolves $LIB to lib.

You can see default library directory definition (BASELIB) here: http://git.openembedded.org/openembedded-core/tree/meta/conf/bitbake.conf#n12

Taras Kondratyuk
  • 26
  • 2
0

On a Debian system I created a mini .so with a ((constructor)) routine
that writes a message and confirmed the .ctor output with this setup:

64-bit lib:   $HOME/lib/x86_64-linux-gnu/ctor.so
LD_PRELOAD='$LIB/ctor.so' /bin/echo

The longer answer in this relate posting on SO provides details with 32- vs 64-bit
and runtime linker use of different subdirs with various distros.

Milag
  • 1,793
  • 2
  • 9
  • 8
  • as I have explained. strace shows it looking for ``` open("/ws/sarvi-sjc/wisktrack/cel8root/lib/libwisktrack.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)open("/ws/sarvi-sjc/wisktrack/cel8root/lib/libwisktrack.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)``` on a 64 bit machine, whch doesnt match anything i am reading – Sarvi Shanmugham Sep 24 '20 at 16:07
  • The above example uses a relative path; related content in the linked posting suggests setting LD_PRELOAD with a relative pathname instead of absolute; maybe provide LD_LIBRARY_PATH as well; might need to adjust your subdirs based on new results from `strace`. What is your Linux distro and version? – Milag Sep 24 '20 at 16:22
  • The build host is RHEL8 and the ld.so seems to map $LIB to lib for 32 bit annd lib64 for 64 bit. for binaries built for the host. But the build system also uses an SDK that yocto/open-embedded. That executables within that extractd sdk seem to be using its own ld.so that seems to map $LIB to lib on that same 64 bit host. So unless I look up what executable I am going to run and know which ld.so it is going to use, from host or from the SDK inside the build system, I can't be sure what $LIB will map to – Sarvi Shanmugham Sep 24 '20 at 18:26