When are header-only libraries acceptable?

Question

Personally, I quite like header-only libraries, but there are claims they cause code bloat due to over-inlining (as well as the other obvious problem of longer compile times).

I was wondering, how much truth is there to these claims (the one about bloat)?

Furthermore, are the costs 'justified'? (Obviously there are unavoidable cases such as when it's a library implemented purely or mostly with templates, however I'm more interested in the case where there's actually a choice available.)

I know there's no hard and fast rule, guideline, etc as far as stuff like this goes, but I'm just trying to get a feel for what others think on the issue.

P.S. Yes this is a very vague and subjective question, I'm aware, and so I have tagged it as such.

Answer 1

In my experience bloat hasn't been a problem:

• Header only libraries give compilers greater ability to inline, but they do not force compilers to inline - many compilers treat the inline keyword as nothing more than a command to ignore multiple identical definitions.

• Compilers usually have options to optimize to control the amount of inlining; /Os on Microsoft's compilers.

• It's usually better to allow the compiler to manage the speed vs. size issues. You'll only see bloat from calls that have actually been inlined, and the compiler will only inline them if its heuristics indicate that it inlining will improve performance.

I wouldn't consider code bloat as a reason to stay away from header only libraries - but I would urge you to consider how much a header only approach will increase compile times by.

Answer 2

I work for a company that has a "Middleware" department of its own to maintain a few hundreds of libraries that are commonly used by a great many teams.

Despite being in the same company, we shy from header only approach and prefer to favor binary compability over performance because of the ease of maintenance.

The general consensus is that the performance gain (if any) would not be worth the trouble.

Furthermore, the so called "code-bloat" may have a negative impact on performance as more code to be loaded in the cache implies more cache miss, and those are the performance killers.

In an ideal world I suppose that the compiler and the linker could be intelligent enough NOT to generate those "multiple definitions" rules, but as long as it is not the case, I will (personally) favor:

• binary compatibility
• non-inlining (for methods that are more than a couple of lines)

Why don't you test ? Prepare the two libraries (one header only and the other without inlining methods over a couple of lines) and check their respective performance in YOUR case.

EDIT:

It's been pointed out by 'jalf' (thanks) that I should precise what I meant exactly by binary compatibility.

2 versions of a given library are said binary compatible if you can (usually) link against one or the other without any change of your own library.

Because you can only link with one version of a given library Target, all the libraries loaded that use Target will effectively use the same version... and here is the cause of the transitivity of this property.

MyLib --> Lib1 (v1), Lib2 (v1)
Lib1 (v1) --> Target (v1)
Lib2 (v1) --> Target (v1)

Now, say that we need a fix in Target for a feature only used by Lib2, we deliver a new version (v2). If (v2) is binary compatible with (v1), then we can do:

Lib1 (v1) --> Target (v2)
Lib2 (v1) --> Target (v2)

However if it's not the case, then we will have:

Lib1 (v2) --> Target (v2)
Lib2 (v2) --> Target (v2)

Yep, you read it right, even though Lib1 did not required the fix, you head to rebuild it against a new version of Target because this version is mandatory for the updated Lib2 and Executable can only link against one version of Target.

With a header-only library, since you don't have a library, you are effectively not binary compatible. Therefore each time you make some fix (security, critical bug, etc...) you need to deliver a new version, and all the libraries that depend on you (even indirectly) will have to be rebuilt against this new version!

Answer 3

I agree, inline libraries are much easier to consume.

Inline bloat mostly depends on the development platform you are working with - specifically, on the compiler / linker capabilities. I wouldn't expect it to be a major problem wiht VC9 except in a few corner cases.

I've seen some notable change in final size in some places of a large VC6 project, but it's hard to give a specific "acceptable, if...". You probalby need to try with your code in your devenv.

A second problem may be compile times, even when using precompiled header (there are tradeoffs, too).

Third, some constructs are problematic - e.g. static data members shared across translation units - or avoiding having a separate instance in each translation unit.

---

I have seen the following mechanism to give the user a choice:

// foo.h
#ifdef MYLIB_USE_INLINE_HEADER
#define MYLIB_INLINE inline
#else 
#define MYLIB_INLINE 
#endif

void Foo();  // a gazillion of declarations

#ifdef MYLIB_USE_INLINE_HEADER
#include "foo.cpp"
#endif

// foo.cpp
#include "foo.h"
MYLIB_INLINE void Foo() { ... }

Answer 4

Over-inlining is probably something that should be addressed by the caller, tuning their compiler options, rather than by the callee trying to control it through the very blunt instruments of the inline keyword and definitions in headers. For example GCC has -finline-limit and friends, so you can use different inlining rules for different translation units. What's over-inlining for you may not be over-inlining for me, depending on architecture, instruction cache size and speed, how the function is used, etc. Not that I've ever needed to do this tuning: in practice when it has been worth worrying about that, it has been worth rewriting, but that could be coincidence. Either way, if I'm the user of a library then all else being equal I'd rather have the option to inline (subject to my compiler, and which I might not take up) than be unable to inline.

I think the terror of code bloat from header-only libraries comes more from the worry that the linker won't be able to remove redundant copies of code. So regardless of whether the function is actually inlined at call sites or not, the concern is that you end up with a callable copy of the function (or class) per object file that uses it. I can't remember whether addresses taken to inline functions in different translation units in C++ have to compare equal, but even assuming they do, so that there is one "canonical" copy of the function in the linked code, it doesn't necessarily mean the linker will actually remove the dead duplicate functions. If the function is defined in only one translation unit, you can be reasonably confident there will only be one stand-alone copy per static library or executable that uses it.

I honestly don't know how well-grounded this fear is. Everything I've worked on has either been so tightly memory constrained that we've used inline only as static inline functions so small that we don't expect the inlined version to be noticeably bigger than the code to make a call, and don't mind duplicates, or else so loosely constrained that we don't care about any duplicates anywhere. I've never yet hit the middle ground of looking for and counting duplicates on various different compilers. I've occasionally heard from others that it's been a problem with template code, though, so I believe that there is truth in the claims.

Making this up as I go along now, I think if you ship a header-only library, the user can always mess with it if they don't like it. Write a new header which declares all the functions and a new translation unit which includes the definitions. Functions defined in classes will have to be moved to external definitions, so if you want to support this use without requiring the user to fork your code, you could avoid doing that and supply two headers:

// declare.h
inline int myfunc(int);

class myclass {
    inline int mymemberfunc(int);
};

// define.h
#include "declare.h"
int myfunc(int a) { return a; }

int myclass::mymemberfunc(int a) { return myfunc(a); }

Callers who are worried about code bloat can probably outwit their compiler by including declare.h in all their files, then writing:

// define.cpp
#include "define.h"

They probably also need to avoid whole-program optimisation to be sure the code won't be inlined, but then you can't be sure that even a non-inline function won't be inlined by whole-program optimisation.

Callers who aren't worried about code bloat can use define.h in all their files.