19

I am working on a large-scale software infrastructure in C++11 that makes extensive use of variadic templates. My question is the following: what is the scalability of this approach? First, is there an upper limit in the number of arguments that variadic templates can take? Second, is code bloat a major issue with state-of-the-art compilers when many arguments are used (and, by extension, many combinations of these arguments that would yield to many different implementations of the templated methods)?

igleyy
  • 605
  • 4
  • 16
Greg
  • 6,038
  • 4
  • 22
  • 37
  • 11
    Theoretically there is no limit, but obviously, there is some limit in practice, dictated by some limited resource like RAM. If you get to a point where that limit matters, you probably have bigger problems, so I wouldn't worry about that one. – R. Martinho Fernandes Sep 27 '13 at 10:23
  • 3
    Your second question is hard to answer, because it depends more on the rest of the code than on exactly how many template arguments there are. If you use many template arguments for calculating stuff compile time, there might not be any code generated. Otherwise, if code is being generated, the number of instantiations will be a multiplicative factor. This is no different than non-variadic templates. – Magnus Hoff Sep 27 '13 at 10:27
  • 13
    Annex B (Implementation quantities), of the C++11 Standard says nothing specifically about *variadic* templates, but it recommends at least 256 function parameters and 1024 template parameters. However, you may run into problems earlier when your mangled names get too long. – Oberon Sep 27 '13 at 10:27
  • 2
    there's a default limit for template parameters used by compilers. you can configure through `-ftemplate-depth-` on clang and gcc. There's also `-fconstexpr-depth=`. I myself was unable to get too far tweaking those two. – oblitum Sep 28 '13 at 05:41
  • 2
    http://cpptruths.blogspot.hu/2010/03/faster-meta-programs-using-gcc-45-and.html – Industrial-antidepressant Oct 09 '13 at 01:23
  • Please give an example of the variadic template usage in question, because all are not created equal. Do you have only more template arguments, or more template instantiations on the whole? Are you passing template instantiations into templates? Doing that recursively? Using template arguments to represent what might be considered "data"? Adding virtual methods to the class produced by all that metaprocessing? Doing that recursively? Etc… – Potatoswatter Nov 14 '13 at 08:23
  • Upvoted. Thanks for the excellent question, was always curious myself. – It'sPete Nov 14 '13 at 22:59
  • I have noticed some recent issues when debugging my code; memory usage with gdb seems to increase dramatically because of the very large number of symbols created by variadic templates. Any idea how to fix this? – Greg Jan 26 '15 at 10:37

1 Answers1

11

I thought I'd have a go at finding out whether there is any limit on the number of template parameters for my particular compiler (g++ 4.8.1 on Linux). I used the following test case:

template <int I>
struct this_is_a_ridiculously_long_struct_name_youd_never_use_in_real_life {};

template <int Depth, typename... T>
struct A
{
    using e = this_is_a_ridiculously_long_struct_name_youd_never_use_in_real_life<Depth>;

    A() {};

    A<Depth - 1, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, e, T...> a;
};

template <typename... T>
struct A<0, T...>
{
};

int main()
{
    A<899> a;
}

The template recursion depth limit defaults to 900 in g++, hence the 899 parameter you see. The ridiculously long struct name is used to see whether I could generate any symbols which were too big for the linker to handle -- more on this in a sec.

In case you can't see what's going on in the test case, basically every instantiation of A creates a member variable which adds 20 extra template parameters. Partial specialisation is used to stop the recursion. By the end, A<0, ...> has somewhere in the region of 18000 template parameters.

What I found was that g++ handled this just fine. It took quite a while to think about it, and used a fair bit of memory, but I was unable to get it to fail simply by increasing the number of template parameters. Clang 3.1 also handled this without any problem once the template recursion depth was set sufficiently (i.e. 900).

Furthermore, although the mangled symbol names do indeed become huge, I was unable to break either nm or ld using them. (It's worth noting that the Linux/Itanium mangling scheme uses substitution so that repeated template parameters of the same type don't repeat the whole type name, but rather are marked S0, S1 etc.) A quick google doesn't seem to turn up any limit on ELF symbol length, but perhaps someone else knows whether such a limit exists.

In conclusion then, for g++ and clang on Linux at least, there doesn't seem to be any practical limit on the number of template parameters.

As to the second part of your question, regarding code bloat, it's very hard to say, particularly once compiler optimisation gets involved. It's easy to do recursion with variadic templates, but then it's easy for the compiler to get rid of intermediate types too. I can only suggest to try it and see.

Tristan Brindle
  • 16,281
  • 4
  • 39
  • 82
  • The number of template arguments I was using (in generated code) depended on the customer and thus was without limit. Thus my answer suggesting to use boost::mpl which avoids this limitation. –  Nov 14 '13 at 18:27