C project build with meson: Correct way to handle 3rd party local libraries

Question

I wanted to build a project that's making use of the Criterion testing library and I'm using meson to build it all. I'm able to build my own code, a static library which relies on just standard c libraries. The unit tests rely on #include <criterion/criterion.h>. If I try to download the binary archive and include it in the meson.build file, I'm not able to sucessfully build the project. I believe I'm not confuring meson correctly. I'm new to meson so I'm not sure what it is I'm getting wrong.

meson.build,

project('is_substring', 'c')

static_library(
    'is_substring',
    'is_substring.h',
)

executable(
    'test_is_substring',
    'is_substring.test.c',
    include_directories: include_directories(
        'libs/criterion-v2.3.3',
    ),
    #dependencies: [
    #    dependency('criterion')
    #]
)

The project structure is,


  ▸ builddir/
  ▸ libs/criterion-v2.3.3/
    is_substring.h
    is_substring.test.c
    meson.build

and the Criterion binary archive tar.bz2 has extracted into ./libs,

  ▾ libs/criterion-v2.3.3/
    ▾ include/criterion/
      ▸ internal/
        abort.h
        alloc.h
        assert.h
        criterion.h
        event.h
        hooks.h
        logging.h
        options.h
        output.h
        parameterized.h
        redirect.h
        stats.h
        theories.h
        types.h
    ▾ lib/
        libcriterion.so
        libcriterion.so.3
        libcriterion.so.3.1.0
    ▾ share/pkgconfig/
        criterion.pc

is_substring.test.c,

#include "criterion/criterion.h"
#include <stdbool.h>
#include <stdio.h>

#include "is_substring.h"


Test(is_substring, is_substring)
{
    cr_assert(is_substring("aaa", "a") == true);
}

is_substring.h,

#include "stdbool.h"

bool is_substring(const char *fullstr, const char *substr)
{
    return true;
}

The error when I try to build is,

francium@4ab5198f934f:/mnt/c/brute-force/builddir$ ninja
[1/2] Compiling C object 'test_is_substring@exe/is_substring.test.c.o'.
FAILED: test_is_substring@exe/is_substring.test.c.o
cc -Itest_is_substring@exe -I. -I.. -I../libs/criterion-v2.3.3 -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -g -MD -MQ 'test_is_substring@exe/is_substring.test.c.o' -MF 'test_is_substring@exe/is_substring.test.c.o.d' -o 'test_is_substring@exe/is_substring.test.c.o' -c ../is_substring.test.c
../is_substring.test.c:1:10: fatal error: criterion/criterion.h: No such file or directory
 #include <criterion/criterion.h>
          ^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
ninja: build stopped: subcommand failed.
francium@4ab5198f934f:/mnt/c/brute-force/builddir$

As pointed out by @jonathan-leffler in the comments, this gcc command works, compiles and links correctly,

gcc -o is_substring.test is_substring.test.c -I libs/criterion-v2.3.3/include/ -Llibs/criterion-v2.3.3/lib -lcriterion

Does it work any better if you change `include_directories: include_directories( 'libs/criterion-v2.3.3', ),` into `include_directories: include_directories( 'libs/criterion-v2.3.3/include', ),`? The thinking is that the include directory is so-named under `libs/criterion-v2.3.3`. — Jonathan Leffler, Mar 28 '20 at 19:45
@JonathanLeffler Yes, then it manages to compile, but during linking it's not able to find the `.so` files. I couldn't figure out how to tell meson where to look for the linkable library files. I thought if I'd just point it to just the whole directory and not just `include/` it would figure it out by convention. — francium, Mar 28 '20 at 19:58
Apparently not. I'd not heard of Meson before I encountered your question, so I don't have any real help I can offer beyond the hackneyed but both time-honoured and surprisingly often relevant advice: "When all else fails, read the manual". I'd expect a 'rule' such as `library_directories` parallel to `include_directories`, but that is pure guesswork. — Jonathan Leffler, Mar 28 '20 at 20:01
There are a few `link_*` options, `link_with`, `link_args`, `link_whole`. Unfortunately might be a bit of trial and error here as there is no nice simple complete example of what I'm trying to do. — francium, Mar 28 '20 at 20:05
@JonathanLeffler What's the gcc command to get this to work? Maybe I can work backwards from there. I tried `gcc -o test is_substring.test.c -I libs/criterion-v2.3.3/include/` and I managed to get the same linker errors. But if I try to add a `-l` (lower case L) or a `-L` neither is working, "/usr/bin/ld: cannot find -l./libs/criterion-v2.3.3/lib/libcriterion.so" — francium, Mar 28 '20 at 20:13
The manual on [`executable()`](https://mesonbuild.com/Reference-manual.html#executable) suggests that you could use `link_args` or `link_with`. You are building the Criterion library within the project (rather than using a version installed in a system directory), so `link_with` may be more appropriate. I guess you'd use `link_args : [ '-Llibs/criterion-v2.3.3/libs', '-lcriterion' ]` but I'm only scanning the documentation at very high speed and guessing. — Jonathan Leffler, Mar 28 '20 at 20:14
On the basis of what I see, you should use `gcc … -Llib/criterion-v2.3.3/lib -lcriterion …`. The library arguments should appear after the last object file. If your own library calls into the Criterion library, it should precede the Criterion library on the command line. If they're independent, the order shouldn't be critical. — Jonathan Leffler, Mar 28 '20 at 20:15
Nice. `-lcriterion ./libs/criterion-v2.3.3/lib/libcriterion.so` works! — francium, Mar 28 '20 at 20:16
Thanks for the insight on the ordering. Not very experience with gcc, so that's very good to know. — francium, Mar 28 '20 at 20:18
Nominally, your suggested command line with `-lcriterion ./libs/…/libcriterion.so` links the same library twice. — Jonathan Leffler, Mar 28 '20 at 20:18
Oh. That's also good to know. I'll RTFM the gcc manual on this a bit more. Thanks for pointing that out too. — francium, Mar 28 '20 at 20:19
On ordering: the `-L` options need to proceed the `-l` options that depend on that location. It is generally wisest to have all object (or source) files listed before any libraries. There are sometimes reasons to do it differently, but not often. That then works with static or shared libraries. Sometimes things work in different orders; sometimes they don't. You do need to have a DAG in the libraries (no cyclic dependencies between libraries) if you're going to be successful with static libraries. Shared libraries can hide ordering problems. There can be compiler/linker options to help. — Jonathan Leffler, Mar 28 '20 at 20:21
Managed to actually get it to link, turns out there is a `objects` argument you can give to the `executable`. But it looks the `so` was dynamically linked and it's unable to find it at runtime. — francium, Mar 28 '20 at 20:43
That is the downside of shared libraries — on Windows, it's known as "DLL Hell". You may need to specify `-R/opt/criterion-v2.3.3/lib` or `-R/usr/local/lib` if you will install your Criterion library into one of those places. Or that may be misremembered from Solaris and not relevant on Linux. The system might look in `/usr/local/lib` anyway. There's also `LD_LIBRARY_PATH`; you could add `/home/you/project/libs/criterion-v2.3.3/lib` (or thereabouts) to your `LD_LIBRARY_PATH` environment variable and that should allow it to be picked up until it is formally installed. — Jonathan Leffler, Mar 28 '20 at 20:47
Right now I'm doing something a bit unorthodox. I've got a docker container running the meson and gcc. I can go through the package manager to install criterion globally, but I wanted to see if I could link against a downloaded binary. More for curiosity and learn about meson — francium, Mar 28 '20 at 20:50
Is it possible to link against the `.so` statically? Using gcc? If gcc can, I can try to work backwards like before and see how meson does it. — francium, Mar 28 '20 at 20:51
I don't think you can link against the `.so` statically. If you have the object files used to create the shared object, you could create a regular static library: `ar -ruv libcriterion-v2.3.3.a *.o` (or specify which object files more precisely). I usually use the `v` option (verbose); the `r` and `u` replace/update files in the library if it exists. Sometimes I see `-rc` (replace, create), but the create happens anyway. You can then link with that `.a` file. — Jonathan Leffler, Mar 28 '20 at 20:55
I think I'll have to subproject criterion into my project and build it statically from source... hmm, seems a bit too much given I'm just trying to run a unit test against some throw away code for fun. I'll post my finding and maybe close this question and add in the bit about specifying the `-R` path to make it work at runtime. Thanks for your help today @JonathanLeffler — francium, Mar 28 '20 at 20:56

score 2 · Answer 1 · answered Mar 28 '20 at 21:02

Managed to get it to compile, but since the Criterion library is a shared library and it's not installed globally, it doesn't work if you just try to run the compiled test executable.

But the fix to make it compile is adding in a,

executable(
    ...
    objects: [
        'libs/criterion-v2.3.3/lib/libcriterion.so',
    ]

Note that the include_directories path should also be 'libs/criterion-v2.3.3/include/' with the include/ at the end.

Full meson.build,

project('is_substring', 'c')

static_library(
    'is_substring',
    'is_substring.h',
)

include_criterion = include_directories(
    'libs/criterion-v2.3.3/include/',
)

executable(
    'test_is_substring',
    'is_substring.test.c',
    include_directories: [
        include_criterion,
    ],
    objects: [
        'libs/criterion-v2.3.3/lib/libcriterion.so',
    ]
)

But running the output executable gives the following error at runtime,

./test_is_substring: error while loading shared libraries: libcriterion.so.3: cannot open shared object file: No such file or directory

As suggested by Jonathan, fixed for this are,

You may need to specify -R/opt/criterion-v2.3.3/lib or -R/usr/local/lib if you will install your Criterion library into one of those places. Or that may be misremembered from Solaris and not relevant on Linux. The system might look in /usr/local/lib anyway. There's also LD_LIBRARY_PATH; you could add /home/you/project/libs/criterion-v2.3.3/lib (or thereabouts) to your LD_LIBRARY_PATH environment variable and that should allow it to be picked up until it is formally installed

Thanks to @jonathan-leffler for his help and insight.

C project build with meson: Correct way to handle 3rd party local libraries

1 Answers1