How do I extend the fusion container size limits beyond 50?

Question

I want to generate boost fusion type sequences with more than 50 elements. The contents of boost/fusion/container/vector/vector50.hpp seems to suggest that a macro BOOST_FUSION_DONT_USE_PREPROCESSED_FILES might be used to affect this limit somehow.

I have created the following simple program which pushes back an int onto a boost::fusion::vector type for a specified number of times, and then converts the result to a vector (which triggers the error).

#include <boost/fusion/container.hpp>                                              
#include <boost/fusion/algorithm.hpp>                                              

#include <type_traits>                                                             

template <typename Sequence, int N>                                                
struct PushBack                                                                    
{                                                                                  
    using type = typename PushBack<typename boost::fusion::result_of::push_back<Sequence, int>::type, N-1>::type;
};                                                                                 

template <typename Sequence>                                                       
struct PushBack<Sequence, 0>                                                       
{                                                                                  
    using type = Sequence;                                                         
};                                                                                 

int main()                                                                         
{                                                                                  
    using NullVector = boost::fusion::vector<>;                                    
    using Sequence = boost::fusion::result_of::as_vector<typename PushBack<NullVector, 20>::type>::type; // this line triggers the error

    Sequence s;                                                                    
    return 0;                                                                      
}                          

When I run this with -D BOOST_FUSION_DONT_USE_PREPROCESSED_FILES -D FUSION_MAX_VECTOR_SIZE=100 I get a flood of errors that look roughly like this:

.../boost/fusion/container/generation/make_vector.hpp:105:25: error: ‘vector51’ does not name a type .../boost/fusion/container/generation/make_vector.hpp:105:25: error: ‘vector52’ does not name a type .../boost/fusion/container/generation/make_vector.hpp:105:25: error: ‘vector...’ does not name a type

Clearly I'm not doing it right. How do I extend this limit past 50? I need at least 150...

Answer 1

This is a two step approach, firstly define: BOOST_MPL_CFG_NO_PREPROCESSED_HEADERS in your compilation flags, and then also set the size you require with -DBOOST_MPL_LIMIT_VECTOR_SIZE=50 -DFUSION_MAX_VECTOR_SIZE=50 (nearest 10)

EDIT:

You are indeed out of luck, in the headers, even if you don't use preprocessed headers, looks like it's limited to 50 by this chunk of code:

#if (FUSION_MAX_VECTOR_SIZE > 40)
#include <boost/fusion/container/vector/vector50.hpp>
#endif

Which then does the really dumb thing:

#define BOOST_PP_FILENAME_1 <boost/fusion/container/vector/detail/vector_n.hpp>
#define BOOST_PP_ITERATION_LIMITS (41, 50)
#include BOOST_PP_ITERATE()

}}

Hmm.. I don't think there is a way around other than "patching" the above code..

EDIT2: Well, where there is a will to hack, there is apparently a way (without having to modify boost), it does come with a huge disclaimer - test properly, it compiles, and you can access elements beyond 50, that's about all the warranty I'll provide...

#include <iostream>

#define BOOST_MPL_LIMIT_VECTOR_SIZE 50
#define FUSION_MAX_VECTOR_SIZE 60

// This sets us up with mpl vector up to 50
#include <boost/mpl/vector.hpp>

// This adds the missing chunk - you should be able to expand this up to template depth
// This sets up the sequence of vectors (vector1 : T + vector0 etc.
#include <boost/fusion/sequence/intrinsic/begin.hpp>
namespace boost { namespace mpl {

#   define BOOST_PP_ITERATION_PARAMS_1 \
    (3,(51, 60, <boost/mpl/vector/aux_/numbered.hpp>))
#   include BOOST_PP_ITERATE()

}}
// This sets up the specializations
#define AUX778076_SEQUENCE_BASE_NAME vector
#   define AUX778076_SEQUENCE_LIMIT BOOST_MPL_LIMIT_VECTOR_SIZE
#   define AUX778076_SEQUENCE_CONVERT_CN_TO(z,n,TARGET) TARGET(BOOST_PP_CAT(C,n))
#   include <boost/mpl/aux_/sequence_wrapper.hpp>

// Include everuthing up to vector50
#define BOOST_FUSION_DONT_USE_PREPROCESSED_FILES
#include <boost/fusion/container/vector/vector50.hpp>

// Add the missing range
namespace boost
{
namespace fusion
{
struct vector_tag;
#define BOOST_PP_FILENAME_1 <boost/fusion/container/vector/detail/vector_n.hpp>
#define BOOST_PP_ITERATION_LIMITS (51, 60)
#include BOOST_PP_ITERATE()
}
}

// Declare the vector class using the FUSION_MAX_VECTOR_SIZE, as the types themselves have been declared above, all is
// good in the world of fusion
#include <boost/fusion/container/vector/vector.hpp>


// Test access
#include <boost/fusion/sequence/intrinsic/at_c.hpp>


int main()
{

  using Sequence = typename boost::fusion::vector<
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    int,
    double,
    std::string,
    std::string
 >;

 Sequence s;

 std::cout << boost::fusion::at_c<51>(s) << std::endl;

}

Basically, all I'm doing above is filling in the missing bits (for counts bigger than 50) but instantiating the bits of mpl and fusion as needed. The above sequence of operations is pretty rigid, if anyone can distill it further - please update...

(NOTE: tested with gcc-4.8.2, boost-1.55, c++11)

Answer 2

Looks to me the pre-processed files are merely a compilation time optimization.

In this case you'd probably want (?) to boldly go where no one has gone before and either:

• edit the limits in that file:

  namespace boost { namespace fusion
  {
      struct vector_tag;
      struct fusion_sequence_tag;
      struct random_access_traversal_tag;
  
  // expand vector41 to vector50
  #define BOOST_PP_FILENAME_1 <boost/fusion/container/vector/detail/vector_n.hpp>
  #define BOOST_PP_ITERATION_LIMITS (41, 100)
  #include BOOST_PP_ITERATE()
  
  }}
  

• add more headers in similar vein, e.g. a vector60.hpp, vector70.hpp... and include it in the proper places

I haven't tested this, but it's worth a try

Answer 3

It does not appear to be possible to extend this limit without either hacking into boost or using the boost wave preprocessor, neither of which are feasible options for me. The only real solution here is to adapt std::tuple to use mpl features and use that instead of fusion containers. This answer shows exactly how to do that:

https://stackoverflow.com/a/15865204/1613983