Why does g++ look in LIBRARY_PATH/../lib64 and where is this documented?

Question

My LIBRARY_PATH environment variable has a custom directory in it: /cs/public/lib/pkg/opencv/lib.

But, when I use g++ --print-search-dirs, I get this instead:

libraries: =
/cs/public/lib/pkg/opencv/lib/x86_64-suse-linux/4.6/:
/cs/public/lib/pkg/opencv/lib/../lib64/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/../lib64/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../x86_64-suse-linux/4.6/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../lib64/:
/lib/x86_64-suse-linux/4.6/:
/lib/../lib64/:
/usr/lib/x86_64-suse-linux/4.6/:
/usr/lib/../lib64/:
/cs/public/lib/pkg/opencv/lib/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../../x86_64-suse-linux/lib/:
/usr/lib64/gcc/x86_64-suse-linux/4.6/../../../:
/lib/:
/usr/lib/

Why does g++ look in these alternatives and a whole bunch of other system locations before what I explicitly specify in the LIBRARY_PATH variable, and where is this documented?

I would understand if system defaults were searched before LIBRARY_PATH and LIBRARY_PATH/../lib64, etc, but g++ puts LIBRARY_PATH/../lib64, then system paths, then LIBRARY_PATH. Where is this ordering documented?

My g++ version is g++ (SUSE Linux) 4.6.2

My OS is openSUSE 12.1 (x86_64)

Answer 1

A similar question was asked here: g++ searches /lib/../lib/, then /lib/

These scary-looking search paths are determined at least in part when the compiler itself it built, for example during the configure phase. It's clear that it goes beyond environment variables because it's possible to have multiple copies of GCC installed and have each of them give different results for gcc --print-search-dirs. Also noting that g++ --print-search-dirs and gcc --print-search-dirs give different results points out that the g++ wrapper is also affecting the search path. Besides configure/build time differences, GCC is definitely aware of the path where its own executable is, and will search subdirectories of that path. A lot of this alchemy can be found in the GCC documentation:
http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/Directory-Options.html#Directory-Options
http://gcc.gnu.org/onlinedocs/gcc-4.7.1/gcc/Environment-Variables.html#Environment-Variables

As far as I know, the most forceful thing that you can without compiling your own copy of GCC is to specify your custom libraries using the -L option. The reason I say this is because -L is searched before e.g. LIBRARY_PATH (see the above link on environment variables). In order to make it more tolerable you could add an alias for g++ including the -L option in your .bashrc file.

If you want a definitive answer then downloading a copy of the GCC source code is one way. For example, in gcc.c the following highly suggestive comment appears:

/* Build a list of search directories from PATHS.
   PREFIX is a string to prepend to the list.
   If CHECK_DIR_P is true we ensure the directory exists.
   If DO_MULTI is true, multilib paths are output first, then
   non-multilib paths.
   This is used mostly by putenv_from_prefixes so we use `collect_obstack'.
   It is also used by the --print-search-dirs flag.  */

However the function that follows the comment is not very obvious.

Answer 2

This is multilib at work - a mechanism which allows for having libraries (but also the whole compilation and build toolchain) for multiple architectures on a single machine. This Wiki states that "The multilib suffix is appended to all directories searched for libraries by GCC and passed via -L options to the linker. The linker itself does not have any particular knowledge of multilibs, and will continue to consult its default search directories if a library is not found in the -L paths. If multiple orthogonal ABI-changing options are used in a single compilation, multiple multilib suffixes can be used in series.".

So, according to the above description, the architecture marker string or different variants thereof are appended to each library search path the compiler receives since it doesn't differentiate between default and custom paths. Your custom path is first in the row, but it undergoes the same "expansion" process as other paths.

Due to the need to handle i386 compatibility, multilib mechanisms seem to now be used by default on most x64 distros, which in practice means most of the installations out there.

Answer 3

I have the exact same problem on:

Fedora 17, gcc 4.7 and gcc 4.3
CentOS 6.3, gcc 4.4
Unubuntu 12, gcc 4.6

So it looks like this is a problem with most gcc versions. Probably this strange behavior first appeared in gcc 4.2 at least according to this.

I tried duping the specs and playing with them. It looks like the *multilib spec is used to append specific strings depending on the platform. For example my original spacs looked like:

*multilib:
. !m64 !m32;64:../lib64 m64 !m32;32:../lib !m64 m32;

When I changed 64:../lib64 to 64:../lib then instead of ../lib64 gcc appended ../lib. But I couldn't fully decipher the meaning of *multilib or any of the other specs.

Answer 4

This answer attempts to summarize the search path behaviors for both GCC and Clang.

GCC

Include path: for a command-line of the following format:

CPLUS_INCLUDE_PATH=EDIR g++ -IIDIR -isystemSDIR

the following list of directories are used as search paths for #include <...>:

IDIR                                           # '-I' directories.
SDIR                                           # '-isystem' directories.
EDIR                                           # *_INCLUDE_PATH directories.
GCCDIR/include/c++/GCCVER                      # libstdc++ directory (C++).
GCCDIR/include/c++/GCCVER/GCCARCH              # libstdc++ directory (C++).
GCCDIR/include/c++/GCCVER/backward             # libstdc++ directory (C++).
GCCDIR/lib/gcc/GCCARCH/GCCVER/include          # GCC arch-specific directory.
/usr/local/include/GCCARCH                     # Local arch-specific include directory.
/usr/local/include                             # Local include directory.
GCCDIR/include                                 # GCC include directory.
GCCDIR/lib/gcc/GCCARCH/GCCVER/include-fixed    # GCC include-fixed directory.
/usr/include/GCCARCH                           # System include arch-specific directory.
/usr/include                                   # System include directory.

Library path: for a command-line of the following format:

LIBRARY_PATH=EDIR gcc -BBDIR -LLDIR

the following arguments are passed to the linker:

-LLDIR                                         # '-L' directories.
-LBDIR                                         # '-B' directories.
-LEDIR/../libXX                                # Multilib directories from LIBRARY_PATH.
-LGCCDIR/lib/gcc/GCCARCH/GCCVER                # GCC arch-specific library directory.
-LGCCDIR/libXX                                 # GCC multilib library directory.
-L/libXX                                       # System multilib library directory.
-L/usr/libXX                                   # System multilib library directory. 
-LEDIR                                         # LIBRARY_PATH directories.
-LGCCDIR/lib                                   # Other GCC libraries.

Clang

Include path: for a command-line of the following format:

CPLUS_INCLUDE_PATH=EDIR clang++ --gcc-toolchain=GCCDIR -BBDIR -IIDIR -isystemSDIR

the following list of directories are used as search paths for #include <...>:

IDIR                                           # '-I' directories.
SDIR                                           # '-isystem' directories.
EDIR                                           # *_INCLUDE_PATH directories.
 # If -stdlib=libstdc++ is used:
   GCCDIR/include/c++/GCCVER                   # libstdc++ directory from the selected GCC toolchain (C++).
   GCCDIR/include/c++/GCCVER/GCCARCH           # libstdc++ directory from the selected GCC toolchain (C++).
   GCCDIR/include/c++/GCCVER/backward          # libstdc++ directory from the selected GCC toolchain (C++).
 # If -stdlib=libc++ is used:
   CLANGDIR/include/c++/v1                     # libc++ directory (C++).
/usr/local/include                             # Local include directory.
CLANGDIR/lib/clang/CLANGVER/include            # Clang include directory.
/include                                       # System include directory.
/usr/include                                   # System include directory.

Library path: for a command-line of the following format:

LIBRARY_PATH=EDIR clang --gcc-toolchain=GCCDIR -BBDIR -LLDIR    

the following arguments are passed to the linker:

-LLDIR                                         # '-L' directories.
-LGCCDIR/lib/gcc/GCCARCH/GCCVER                # GCC arch-specific library directory.
-LGCCDIR/libXX                                 # GCC multilib library directory.
-L/libXX                                       # System multilib library directory.
-L/usr/libXX                                   # System multilib library directory.
-LGCCDIR/lib                                   # Other GCC libraries.
-LCLANGDIR/lib                                 # Clang libraries.
-L/lib                                         # System library directory.
-L/usr/lib                                     # System library directory.
-LEDIR                                         # LIBRARY_PATH directories.

Summary

The search path for includes are pretty much the same in both GCC and Clang. C++-specific paths are omitted if the C frontend is used in both cases. Library search paths differ substantially between GCC and Clang, notably the presence of -B directories and the odd manipulation of LIBRARY_PATH in the GCC frontend.

The library search paths are the same for both C and C++ frontends. Other library search paths are introduced by the linker itself. The following excerpt comes from the vanilla linker script for GNU Binutils:

# Multilib library directories.
SEARCH_DIR("BINUTILSDIR/BINUTILSARCH/libXX");
SEARCH_DIR("BINUTILSDIR/libXX");
SEARCH_DIR("/usr/local/libXX");
SEARCH_DIR("/libXX");
SEARCH_DIR("/usr/libXX");
# Traditional library directories.
SEARCH_DIR("BINUTILSDIR/BINUTILSARCH/lib");
SEARCH_DIR("BINUTILSDIR/lib");
SEARCH_DIR("/usr/local/lib");
SEARCH_DIR("/lib");
SEARCH_DIR("/usr/lib");

It is also imperative to note that library dependencies are not searched for inside the directories listed above. These exclusively rely on -rpath and -rpath-link options passed to the linker, or else they get resolved from the default system library paths. Thus it might be useful to produce both -L and -rpath-link arguments in order to guarantee that the correct libraries are linked.

Finally, special files (such as the CRT objects) are searched for only in -B directories. In Clang, special files are also searched inside the selected GCC toolchain. Other factors (spec files, distribution-specific configuration) might change some or all of the above.

Answer 5

Looks like it's needed for cross-compiling. From the ChangeLog:

  Wed Mar 29 14:53:23 1995  Jim Wilson  <wilson@cygnus.com>

          * gcc.c (process_command): Delete code modifying gcc_exec_prefix.
          (main): Put it here after last use of gcc_exec_prefix.  For cross
          compiler, set startfile_prefixes if gcc_exec_prefix is set and
          standard_startfile_prefix is a relative path.

startfile_prefixes is what is being printed out with the search-dirs flags. From gcc/gcc.c:

    if (print_search_dirs)
      {
        printf (_("install: %s%s\n"), standard_exec_prefix, machine_suffix);
        printf (_("programs: %s\n"), build_search_list (&exec_prefixes, "", 0));
        printf (_("libraries: %s\n"), build_search_list (&startfile_prefixes, "", 0));
        return (0);
      }

Answer 6

The compiler will first look at the default paths then for the others. How does it sorted when you print I don't now, but it is documented here, 3.19 Environment Variables Affecting GCC.

Answer 7

The paths are defined by the built-in specs. Specs define how the pipeline processes the source code to obtain a result. GCC just drives the compilation.

You can give GCC your own spec file via -spec=, and you can get the built-in specs with -dumpspecs IIRC.

This is probably explained somewhere in the GCC manual.