gcc and ld is refusing to link library statically

Question

Just for fun (and for learning compiling in general), I gave myself the task to build a xorg-server with all libraries statically linked, except for glibc. For some reason, gcc simply ignores some of my static libraries.

When I run something like this (last 2 lines are the relevant ones):

gcc -DHAVE_DIX_CONFIG_H -Wall -Wpointer-arith -Wmissing-declarations -Wformat=2 -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Wbad-function-cast \
-Wold-style-definition -Wdeclaration-after-statement -Wunused -Wuninitialized -Wshadow -Wmissing-noreturn -Wmissing-format-attribute -Wredundant-decls \
-Wlogical-op -Werror=implicit -Werror=nonnull -Werror=init-self -Werror=main -Werror=missing-braces -Werror=sequence-point -Werror=return-type -Werror=trigraphs \
-Werror=array-bounds -Werror=write-strings -Werror=address -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -fno-strict-aliasing -fno-strict-aliasing \
-D_DEFAULT_SOURCE -D_BSD_SOURCE -DHAS_FCHOWN -DHAS_STICKY_DIR_BIT -I/usr/include/libdrm -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng16 \
-I/usr/include/X11/dri -I../../include -I../../include -I../../Xext -I../../composite -I../../damageext -I../../xfixes -I../../Xi -I../../mi -I../../miext/sync \
-I../../miext/shadow -I../../miext/damage -I../../render -I../../randr -I../../fb -I../../dbe -I../../present -fvisibility=hidden -DHAVE_XORG_CONFIG_H \
-fvisibility=hidden -I/usr/include/libdrm -g -O2 -pthread -o Xorg sdksyms.o \
   /lib/arm-linux-gnueabihf/libz.a ../../dix/.libs/libmain.a ../../dix/.libs/libdix.a loader/.libs/libloader.a common/.libs/libcommon.a  os-support/.libs/libxorgos.a \
parser/.libs/libxf86config.a dixmods/.libs/libdixmods.a modes/.libs/libxf86modes.a ramdac/.libs/libramdac.a ddc/.libs/libddc.a i2c/.libs/libi2c.a \
../../composite/.libs/libcomposite.a ../../xfixes/.libs/libxfixes.a ../../Xext/.libs/libXext.a ../../dbe/.libs/libdbe.a ../../record/.libs/librecord.a \
../../randr/.libs/librandr.a ../../render/.libs/librender.a ../../damageext/.libs/libdamageext.a ../../present/.libs/libpresent.a \
../../miext/damage/.libs/libdamage.a ../../Xi/.libs/libXi.a ../../xkb/.libs/libxkb.a xkb/.libs/libxorgxkb.a dri/.libs/libdri.a dri2/.libs/libdri2.a \
../../dri3/.libs/libdri3.a ../../glx/.libs/libglxvnd.a ../../miext/sync/.libs/libsync.a ../../mi/.libs/libmi.a ../../os/.libs/libos.a -lcrypto \
../../Xext/.libs/libXvidmode.a \
-Wl,-Bstatic -lpciaccess -ldrm -lpixman-1 -lXau -lxshmfence -lm -lbz2 -lfontenc -ludev \
-Wl,-Bdynamic -lXdmcp -lXfont2 -lrt -ldl -lpthread

It produces a binary with the following links:

(sid)root@localhost:/opt/xorg-server-1.20.5/hw/xfree86# ldd Xorg
        libcrypto.so.1.1 => /lib/arm-linux-gnueabihf/libcrypto.so.1.1 (0xf6a8b000)
        libXdmcp.so.6 => /lib/arm-linux-gnueabihf/libXdmcp.so.6 (0xf6a77000)
        libXfont2.so.2 => /lib/arm-linux-gnueabihf/libXfont2.so.2 (0xf6a4b000)
        librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0xf6a35000)
        libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xf6a22000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xf69fd000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xf6902000)
        /lib/ld-linux-armhf.so.3 (0xf6ec7000)
        libbsd.so.0 => /lib/arm-linux-gnueabihf/libbsd.so.0 (0xf68df000)
        libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0xf68bc000)
        libbz2.so.1.0 => /lib/arm-linux-gnueabihf/libbz2.so.1.0 (0xf68a0000)
        libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xf6837000)
        libfontenc.so.1 => /lib/arm-linux-gnueabihf/libfontenc.so.1 (0xf6822000)
        libfreetype.so.6 => /lib/arm-linux-gnueabihf/libfreetype.so.6 (0xf67a2000)
        libpng16.so.16 => /lib/arm-linux-gnueabihf/libpng16.so.16 (0xf6770000)

As seen on the output, "libz" is dynamically linked. So I thought I would try to force gcc/ld to use the static libz instead. But nothing works. I have tried adding "-lz" to the -Bstatic list and I have also tried with "-l:libz.a", as well as with the full path to the file "/lib/arm-linux-gnueabihf/libz.a". Nothing works and the final binary still links to the dymanic .so library. gcc does not give me any errors, it simply seem to ignore my request of statically linking some libraries.

What am I doing wrong?

Also, when I move the two libraries "-lXdmcp -lXfont2" from the dynamic list to the static list, gcc fails with some errors that looks like the libraries are referencing things that is not included? How would I debug linking problems like that?

/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXdmcp.a(Key.o): in function `XdmcpGenerateKey':
(.text+0x2): undefined reference to `arc4random_buf'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `sfnt_get_ushort':
(.text+0x1a): undefined reference to `FT_Load_Sfnt_Table'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `FreeTypeFreeFace.part.3':
(.text+0x412): undefined reference to `FT_Done_Face'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `FreeTypeFreeFont':
(.text+0x4aa): undefined reference to `FT_Done_Size'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `FreeTypeActivateInstance':
(.text+0x686): undefined reference to `FT_Activate_Size'
/usr/bin/ld: (.text+0x6b4): undefined reference to `FT_Set_Transform'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `FreeTypeLoadFont':
(.text+0x8f8): undefined reference to `FT_Get_Sfnt_Table'
/usr/bin/ld: (.text+0xb6e): undefined reference to `FT_Get_PS_Font_Info'
/usr/bin/ld: (.text+0xc7c): undefined reference to `FT_New_Size'
/usr/bin/ld: (.text+0xcb8): undefined reference to `FT_Set_Char_Size'
/usr/bin/ld: (.text+0xcc6): undefined reference to `FT_Done_Size'
/usr/bin/ld: (.text+0xe70): undefined reference to `FT_Set_Pixel_Sizes'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `FT_Do_SBit_Metrics.isra.6':
(.text+0xf9c): undefined reference to `FT_Set_Pixel_Sizes'
/usr/bin/ld: (.text+0xfaa): undefined reference to `FT_Load_Glyph'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `ft_get_very_lazy_bbox':
(.text+0x10a4): undefined reference to `FT_Load_Sfnt_Table'
/usr/bin/ld: (.text+0x10d8): undefined reference to `FT_MulFix'
/usr/bin/ld: (.text+0x10e8): undefined reference to `FT_MulFix'
/usr/bin/ld: (.text+0x10f2): undefined reference to `FT_MulFix'
/usr/bin/ld: (.text+0x10fc): undefined reference to `FT_MulFix'
/usr/bin/ld: (.text+0x116a): undefined reference to `FT_Vector_Transform'
/usr/bin/ld: (.text+0x1172): undefined reference to `FT_Vector_Transform'
/usr/bin/ld: (.text+0x117a): undefined reference to `FT_Vector_Transform'
/usr/bin/ld: (.text+0x1182): undefined reference to `FT_Vector_Transform'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `FreeTypeRasteriseGlyph':
(.text+0x1530): undefined reference to `FT_Outline_Get_BBox'
/usr/bin/ld: (.text+0x1704): undefined reference to `FT_Outline_Get_BBox'
/usr/bin/ld: (.text+0x17d0): undefined reference to `FT_Load_Glyph'
/usr/bin/ld: (.text+0x1810): undefined reference to `FT_Outline_Get_BBox'
/usr/bin/ld: (.text+0x1c56): undefined reference to `FT_Load_Glyph'
/usr/bin/ld: (.text+0x1d0e): undefined reference to `FT_Render_Glyph'
/usr/bin/ld: (.text+0x1dd0): undefined reference to `FT_Outline_Get_BBox'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftfuncs.o): in function `FreeTypeLoadXFont':
(.text+0x3042): undefined reference to `FT_New_Face'
/usr/bin/ld: (.text+0x3088): undefined reference to `FT_Init_FreeType'
/usr/bin/ld: (.text+0x33f6): undefined reference to `FT_Get_Sfnt_Table'
/usr/bin/ld: (.text+0x3af0): undefined reference to `FT_Get_Sfnt_Table'
/usr/bin/ld: (.text+0x3afc): undefined reference to `FT_Get_Sfnt_Table'
/usr/bin/ld: (.text+0x3b08): undefined reference to `FT_Get_PS_Font_Info'
/usr/bin/ld: (.text+0x3e8a): undefined reference to `FT_Get_Postscript_Name'
/usr/bin/ld: (.text+0x3f04): undefined reference to `FT_Get_X11_Font_Format'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(fttools.o): in function `FTGetName':
(.text+0x22): undefined reference to `FT_Get_Sfnt_Name_Count'
/usr/bin/ld: (.text+0x40): undefined reference to `FT_Get_Sfnt_Name'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(gunzip.o): in function `BufZipFileClose':
(.text+0x48): undefined reference to `inflateEnd'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(gunzip.o): in function `BufZipFileFill':
(.text+0xcc): undefined reference to `inflate'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(gunzip.o): in function `BufFilePushZIP':
(.text+0x1cc): undefined reference to `inflateInit2_'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(bunzip2.o): in function `BufBzip2FileClose':
(.text+0x48): undefined reference to `BZ2_bzDecompressEnd'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(bunzip2.o): in function `BufBzip2FileFill':
(.text+0xce): undefined reference to `BZ2_bzDecompress'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(bunzip2.o): in function `BufFilePushBZIP2':
(.text+0x1c2): undefined reference to `BZ2_bzDecompressInit'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftenc.o): in function `FTPickMapping':
(.text+0x20): undefined reference to `FontEncFromXLFD'
/usr/bin/ld: (.text+0x68): undefined reference to `FT_Get_BDF_Charset_ID'
/usr/bin/ld: (.text+0xd0): undefined reference to `FT_Get_BDF_Charset_ID'
/usr/bin/ld: (.text+0xe2): undefined reference to `FT_Select_Charmap'
/usr/bin/ld: (.text+0xf4): undefined reference to `FontEncFind'
/usr/bin/ld: (.text+0x10a): undefined reference to `FontEncFind'
/usr/bin/ld: (.text+0x118): undefined reference to `FT_Has_PS_Glyph_Names'
/usr/bin/ld: (.text+0x18c): undefined reference to `FT_Get_BDF_Charset_ID'
/usr/bin/ld: (.text+0x19c): undefined reference to `FontEncFind'
/usr/bin/ld: (.text+0x242): undefined reference to `FontEncFind'
/usr/bin/ld: (.text+0x282): undefined reference to `FT_Get_Sfnt_Table'
/usr/bin/ld: /usr/lib/gcc/arm-linux-gnueabihf/9/../../../arm-linux-gnueabihf/libXfont2.a(ftenc.o): in function `FTRemap':
(.text+0x2d6): undefined reference to `FontEncName'
/usr/bin/ld: (.text+0x2ea): undefined reference to `FontEncRecode'
/usr/bin/ld: (.text+0x2f6): undefined reference to `FT_Set_Charmap'
/usr/bin/ld: (.text+0x316): undefined reference to `FT_Set_Charmap'
/usr/bin/ld: (.text+0x2e6): undefined reference to `FT_Get_Name_Index'
/usr/bin/ld: (.text+0x304): undefined reference to `FT_Get_Char_Index'
/usr/bin/ld: (.text+0x326): undefined reference to `FT_Get_Char_Index'
collect2: error: ld returned 1 exit status

score 1 · Answer 1 · answered Nov 01 '19 at 10:37

1

Short Answer: If you opt for static libraries (assuming that there are ".a" available), the order of the libraries matter - on the command line matter !

Long Answer: If you have two libraries (A, B) linked with the main (M) code, where there are calls M->A, and A->B, you have to specify the libraries in that order: M, A, B. Recall that when a static library is referenced, the linker will attempt to look for any unresolved reference (function, variable, method, ...) in the named library, and extract the only the '.o' files that resolve those references from the '.a' into the executable.

If the libraries are specified as M, B, A, the linker will look into library 'B' and will not identify any '.o' to be included (since there is only reference M->A). It will then retrieve the '.o' from the A library to satisfy the call M->A, and report error on the newly discovered calls A->B.

When using shared-objects, the whole '.so' is linked (by reference), and all globally defined symbols in the '.so' will be available to any module in the executable, including modules linked from other .so. Therefore, the order of specifying '.so' will usually not matter

answered Nov 01 '19 at 10:37

dash-o

13,723
1
10
37

Not sure how your answer is an actual answer to my question :) Moving the "-lz" to the beginning of the list vs to the end of the list makes no difference. Same goes with the 2 libraries that gives error. I have tried them in all different kind of orders and it still gives the same error. So a follow up question to your answer, how do I find out which order to put the libraries? (even though it will not solve my libz problem). – Daniele Testa Nov 01 '19 at 11:39
If I put something like "-l:foobar.a", gcc will fail with error that it cannot find "foobar.a". But when I do "-l:libz.a", I get no error. But still the binary has linked libz.so dynamically. – Daniele Testa Nov 01 '19 at 12:14
Question has two items (1) libz - it is dynamically linked because one of the other '.so' was built with it - it will be linked into your binary, regardless of the existence of static libz. (2) library order (Xdmcp -lXfont2) - moved forward to the staticpart, which I've tried to explain in the post – dash-o Nov 01 '19 at 13:57
Yes, I managed to figure out that you can run "ldd Xdmcp.so" and see what other dynamic libraries it was loading and therefor needed to be included in the static list as well. – Daniele Testa Nov 01 '19 at 14:05

score 0 · Answer 2 · answered Nov 01 '19 at 12:21

it simply seem to ignore my request of statically linking some libraries.

Unlikely per se.

What am I doing wrong?

You are misunderstanding the nature of dynamic linking. At least these factors are in play:

The shared objects you are linking in may have their own dependencies on other dynamic objects.
Depending on platform and implementation details, the link may well succeed even though some of the dynamic objects' own dependencies are not resolved by anything named on the command line.
The static libraries you are linking very likely do not provide viable resolutions for anything in any of the dynamic libraries.
The overall dynamic dependencies of a program, such as are presented by ldd, is the transitive closure of the dependencies of all shared objects involved. They are not necessarily all referenced directly by the SO that contains the program entry point, and they could even change over time if shared library implementations are swapped out.

Bottom line: it is very likely that at least some of your unexpected dynamic dependencies are coming from the shared objects you are linking, probably libXdmcp and libXfont2. The static libraries are linked, and they will be used to resolve symbols in the main program and in static libraries preceding them on the command line, but they cannot satisfy the shared objects' references to dynamic symbols, no matter where anything appears on the command line.

Also, when I move the two libraries "-lXdmcp -lXfont2" from the dynamic list to the static list, gcc fails with some errors that looks like the libraries are referencing things that is not included? How would I debug linking problems like that?

Static libraries need to be ordered correctly on the link command line. Each one must precede the (static) libraries containing symbols they need, so these particular ones probably need to appear near the front. If necessary, you may repeat libraries on the command line, which is useful if you have circular dependencies, or can serve as a quick hack if you're having trouble finding a viable order otherwise.

You would debug by figuring out with library does provide the symbols the linker complains about, and which library references each. This tells you about the necessary command-line ordering.

gcc and ld is refusing to link library statically

2 Answers2