1

I've been trying to compile against Musl for a while now. My goal is to make a game engine binary which can execute on as many x86_64 Linux distros as possible with the same binary.

The problem is, symbols such as pthread_mutex_init are not found by the linker, x86_64-linux-musl-gcc, even though pthread is embedded into the Musl libc as per Musl design.

I'm building the engine in Rust and am using the command, cargo build --release --target=x86_64-unknown-linux-musl --bin catgirl-engine for debugging the compilation steps.

I've created an Asciinema Recording to demonstrate the compilation failure.

The static error is:

  = note: /home/alexis/Desktop/game/build/x86-64-linux-musl-cross/bin/../lib/gcc/x86_64-linux-musl/9.4.0/../../../../x86_64-linux-musl/bin/ld: /home/alexis/Desktop/game/android/app/jni/SDL/build/libSDL2.a(SDL_sysmutex.c.o): undefined reference to symbol 'pthread_mutex_init'
          /home/alexis/Desktop/game/build/x86-64-linux-musl-cross/bin/../lib/gcc/x86_64-linux-musl/9.4.0/../../../../x86_64-linux-musl/bin/ld: /home/alexis/Desktop/game/build/x86-64-linux-musl-cross/bin/../x86_64-linux-musl/lib/libc.so: error adding symbols: DSO missing from command line
          collect2: error: ld returned 1 exit status

Before I found out that pthread was embedded in Musl's libc, I tried linking to the lib directory as well as specifying the include directory for the headers. I've even tried linking against the files in Debian packages libc6-dev-amd64-cross and musl-dev.

I was expecting to compile the engine without being tied to a specific version of libc. This is to increase mass portability as then I'd just need to ship my engine with the custom version of SDL, SDL_image, and SDL_ttf.

I already successfully build the engine for Android and package SDL, SDL_image, and SDL_ttf with the Android build of the app, so I'd like to do the same with the x86_64 version.

I first discovered this problem when I learned the glibc version on Github Actions is far newer than what Debian provides.

Here's my ~/.cargo/config

[target.x86_64-unknown-linux-musl]
ar = "/home/alexis/Desktop/game/build/x86-64-linux-musl-cross/bin/x86_64-linux-musl-ar"
linker = "/home/alexis/Desktop/game/build/x86-64-linux-musl-cross/bin/x86_64-linux-musl-gcc"
rustflags = ["-L/home/alexis/Desktop/game/android/app/jni/SDL/build", "-L/home/alexis/Desktop/game/android/app/jni/SDL_image/build", "-L/home/alexis/Desktop/game/android/app/jni/SDL_ttf/build", "-L/home/alexis/Desktop/game/build/x86-64-linux-musl-cross/lib", "-L/home/alexis/Desktop/game/build/x86-64-linux-musl-cross/x86_64-linux-musl/lib", "-Clink-arg=-Wl,-rpath,.,-rpath-link,/home/alexis/Desktop/game/android/app/jni/SDL/build", "-Clink-arg=-L/home/alexis/Desktop/game/build/x86-64-linux-musl-cross/x86_64-linux-musl/lib", "-Clink-arg=-lpthread"]

The last build I made of x86_64-linux-musl-cross was based on this commit. As Github Actions (through Ubuntu) uses a newer libc than my laptop (latest release Debian), I am using a local build of x86_64-linux-musl-cross for my local building and am using the Github build of the same commit for remote building.

Alexis
  • 81
  • 7
  • would [this](https://stackoverflow.com/questions/63739813/) help you? – xc wang May 02 '23 at 09:20
  • No, I've already had the correct linker called. It just doesn't want to find the pthread symbols even though they are embedded into the libc library which the linker is pulling the missing symbols from. Musl is weird in that pthread is embedded as opposed to glibc which is separated – Alexis May 02 '23 at 21:17

2 Answers2

0

The error, undefined reference to pthread_mutex_init, indicates that the Musl does not provide pthreads directly.

Musl is supposed to be a 'lightweight' library that supports a minimal set of functionalities. It does include pthreads support, but not as a core library feature, and is instead provided as a separate library, libpthread.

To link pthread functions, you will need to add the -lpthread flag to your linker command:

cargo build --release --target=x86_64-unknown-linux-musl --bin catgirl-engine -C link-args=-lpthread
Jishan Shaikh
  • 1,572
  • 2
  • 13
  • 31
  • The -C arg doesn't exist for this command. When I also attempted to apply this to the RUSTFLAGS variable, it didn't change the results of the compilation. – Alexis May 02 '23 at 13:38
  • Try L: `C link-arg=-L/path/to/musl/lib` – Jishan Shaikh May 02 '23 at 14:11
  • Unfortunately this does not change the output. I've updated the question with my copy of ~/.cargo/config so you can see what arguments I've provided to the linker. – Alexis May 02 '23 at 21:19
  • The description of the state of pthreads in MUSL is misleading. The Pthread API is included in MUSL, period. That you need to link it explicitly doesn't change that -- the same was true of Glibc until very recently, and is true of many C libraries' floating-point functions (libm). With some toolchains you even need to specify the standard library itself explicitly (libc). Having the pthreads functions in a separate file and requiring explicit link options does not make MUSL support fail to be "direct". – John Bollinger May 05 '23 at 14:11
  • @JohnBollinger Your suggestion of explicitly adding libc worked. Now it compiles, I just have to figure out why the function, pthread_mutexattr_init@plt, is causing segfault 14. I can see something about a missing page, but am not sure why when I compile for Musl, it segfaults, but it works fine when compiling for Android and for my laptop's version of glibc. – Alexis May 08 '23 at 12:07
  • It appears the segfault is caused by initializing SDL2. As in `let sdl_context: Sdl = sdl2::init()?;`. I may have to make a separate question if I can't figure this out as I'm not sure where to start with debugging segfaults. – Alexis May 08 '23 at 12:29
0

As per @JohnBollinger's suggestion, what I needed to do was explicitly link the c library. I added the rust flag, -Clink-arg=-Wl,-lc, to the build.

There was other stuff I needed to solve, but they are different problems. The solution I provided in this answer is the correct one to solve this question.

Alexis
  • 81
  • 7