4

To reduce compile times in a template-heavy project, I'm trying to explicitly instantiate many templates in a separate compilation unit. Because these templates depend on enum class members, I'm able to list all possible instantiations. I want all other cpp-files to only see the declaration. While I'm able to do this, I run into problems trying to factorize the explicit instantiations. I will first explain 2 working examples below, in order to explain what exactly my issue is (example 3):

Example 1

/* test.h
   Contains the function-template-declaration, not the implementation.
*/

enum class Enum
{
    Member1,
    Member2,
    Member3
};

struct Type
{
    template <Enum Value>
    int get() const;
};
/* test.cpp
   Only the declaration is visible -> needs to link against correct instantiation.
*/

#include "test.h"

int main() {
    std::cout << Type{}.get<Enum::Member1>() << '\n';
}
/* test.tpp 
   .tpp extension indicates that it contains template implementations.
*/

#include "test.h"

template <Enum Value>
int Type::get() const
{
    return static_cast<int>(Value); // silly implementation
}
/* instantiate.cpp
   Explicitly instantiate for each of the enum members.
*/

#include "test.tpp"

template int Type::get<Enum::Member1>() const;
template int Type::get<Enum::Member2>() const;
template int Type::get<Enum::Member3>() const;

As mentioned before, the above compiles and links without issues. However, in the real application, I have many function-templates and many more enum-members. Therefore, I tried making my life somewhat easier by grouping the members together in a new class, which itself depends on the template parameter and explicitly instantiate this class for each of the enum-values.

Example 2

// instantiate.cpp

#include "test.tpp"

template <Enum Value>
struct Instantiate
{
    using Function = int (Type::*)() const;
    static constexpr Function f1 = Type::get<Value>;
   
    // many more member-functions
};

template class Instantiate<Enum::Member1>;
template class Instantiate<Enum::Member2>;
template class Instantiate<Enum::Member3>;

This still works (because in order to initialize a pointer to a member, this member has to be instantiated), but when the number of enum-members is large, it will still be messy. Now I can finally get to the issue. I thought I could factorize even further by defining a new class-template that depends on a parameter pack, which then derives from each of the types in the pack like so:

Example 3

// instantiate.cpp

#include "test.tpp"

template <Enum Value>
struct Instantiate { /* same as before */ };

template <Enum ... Pack>
struct InstantiateAll:
    Instantiate<Pack> ...
{};

template class InstantiateAll<Enum::Member1, Enum::Member2, Enum::Member3>; 

This should work, right? In order to instantiate InstantiateAll<...>, each of the derived classes have to be instantiated. At least, this is what I thought. The above compiles but results in a linker-error. Upon checking the symbol-table of instantiate.o with nm, it's confirmed that nothing at all has been instantiated. Why not?

Of course, I can get by using example 2, but it really got me curious why things break down like this.

(Compiling with GCC 10.2.0)

Edit: same happens on Clang 8.0.1 (although I have to use the address-of-operator explicitly in assigning the function-pointers: Function f1 = &Type::get<Value>;)

Edit: User 2b-t kindly made the examples available through https://www.onlinegdb.com/HyGr7w0fv_ for people to experiment with.

JorenHeit
  • 3,877
  • 2
  • 22
  • 28
  • Off Topic: are you sure that you need the tpp files? I mean: what about defining (`struct Type { template int get () const { return static_cast(Value); } };`) the methods directly inside test.h ? – max66 Apr 25 '21 at 11:36
  • 1
    I put together your code in an online compiler: https://onlinegdb.com/HyGr7w0fv_ It might be helpful to add it to your description as people can play easily around with it... – 2b-t Apr 25 '21 at 11:46
  • @max66 When multiple other cpp-files are including the header and using the templates, these functions are compiled multiple times. Duplicates are removed by the linker. This results in longer compilation and link-times. – JorenHeit Apr 25 '21 at 12:01
  • @2b-t Thanks! Added your link to the question :) – JorenHeit Apr 25 '21 at 12:03
  • 2
    Not only does moving implementations into a ".tpp" file like this reduce the amount of code user code sees, it also makes the compiler run faster for them and use less memory. The other benefit is that user code has no dependencies on the implementation, or on headers that the implementation code includes, so modifications to implementation do not cause user code to be recompiled - only re-linked. For common headers, especially if edited often, this savings of re-builds can be vast. – Chris Uzdavinis Apr 25 '21 at 12:34
  • 2
    For the technical correctness question, [C++20 \[temp.explicit\]/11](https://timsong-cpp.github.io/cppwp/n4861/temp.explicit#11) says "An explicit instantiation that names a class template specialization is also an explicit instantiation of the same kind (declaration or definition) of each of its members _(not including members inherited from base classes and members that are templates)_ that..." (emphasis mine). (The current draft has simplified the wording to just "direct non-template members".) – aschepler Apr 25 '21 at 13:32
  • So I think this also explains why the answer of 2b-t does work. – aschepler Apr 25 '21 at 13:33
  • @aschepler Does this also explain why adding the attribute (suggested in the accepter answer) works? It's not quite clear to me yet why this would be the case. – JorenHeit Apr 25 '21 at 13:45
  • @JorenHeit No, `__attribute__((used))` is not in the C++ Standard at all, and is a language extension of gcc and clang. The gcc manual does explain it: https://gcc.gnu.org/onlinedocs/gcc-10.3.0/gcc/Common-Variable-Attributes.html, under "used". – aschepler Apr 25 '21 at 14:27

2 Answers2

4

If the compiler sees that code isn't referred to, even for static initialization with side effects, it can eliminate it, and I think that's the case in your example. It can "prove" those class instantiations are not used, and so the side effects are lost.

For a non-standard solution, but one that works on g++ (and presumably clang, but not tested) is to mark your static data members with the "used" attribute:

template <Enum Value>
struct Instantiate
{
    using Function = int (Type::*)() const;
    static constexpr Function f1 __attribute__((used)) = &Type::get<Value>;
   
    // many more member-functions
};

Update

Reviewing the standard, the wording seems like I got it exactly backwards:

"If an object of static storage duration has initialization or a destructor with side effects, it shall not be eliminated even if it appears to be unused, except that a class object or its copy may be eliminated as specified in ..."

So I've had this in my head for decades, and now I'm uncertain as to what I was thinking. :) But it seems related, given the attribute helps. But now I have to learn what's going on.

Chris Uzdavinis
  • 6,022
  • 9
  • 16
  • 1
    Thanks! That indeed works. It's still weird to me, because how can the compiler "prove" that these instantiations are not used in other CU's? – JorenHeit Apr 25 '21 at 13:07
  • 1
    See my standardese comment on the question. Briefly, members of the base classes are not instantiated. – aschepler Apr 25 '21 at 13:36
2

I can't give you yet a good answer to why this does not work (maybe I can do so later or somebody else can) but instead of having Instantiate and InstantiateAll having only a variadic InstantiateAll as follows works

template <Enum ... Pack>
struct InstantiateAll {
  using Function = int (Type::*)() const;
  static constexpr std::array<Function,sizeof...(Pack)> f = {&Type::get<Pack> ...};
};
template class InstantiateAll<Enum::Member1, Enum::Member2, Enum::Member3>;

Try it here.

2b-t
  • 2,414
  • 1
  • 10
  • 18
  • 1
    Thanks! That's a viable solution. I accepted Chris' answer because his answer addresses the reason it's not working right now. – JorenHeit Apr 25 '21 at 13:06
  • 1
    You are welcome. I agree his answer gives a reason why. I have tried a bit and for example if you would make `f1` call the function (e.g. if it was `static constexpr`) instead of having a function pointer it would work as well. I guess just having the pointer but nobody using it makes it think it is unused. Unexpected but very interesting... – 2b-t Apr 25 '21 at 13:10