How Can I Build A Sequence Of Structs With Integer Members of Increasing Value at Compile Time in C++11

Question

Working in an embedded environment, I'm repeatedly writing code that takes an array of bytes from a protocol layer and turns those bytes into a C++ class representation.

An example array of bytes that represents a uint32_t, followed by a uint8_t, followed by a uint16_t might look like this.

std::array<uint8_t, 7> bytes(0x01, 0x02, 0x03, 0x04, 0x10, 0x20, 0x30);

Where 0x01020304 is my uin32_t, 0x10 is my uint8_t and 0x2030 is my uint16_t.

I also have a variadic function func() that I want to call with the values parsed out of the payload.

To achieve this, I manually define an intermediate object:

// Declaring the Object
struct MY_TYPE
{
   uint32_t val1;
   uint8_t val2;
   uint16_t val3;
} __attribute__((__packed__));

// Processing the Bytes 
auto & object(reinterpret_cast<MY_TYPE *>(&bytes));

func(object.val1, object.val2, object.val3)

What I want to do is implement a variadic class such that I don't need to re-implement MY_TYPE for every combination of types.

Here's what I initially tried:

template <typename... Types>
struct GENERIC_CLASS
{
   template <typename ReturnType, std::size_t ArraySize>
   ReturnType getValueFromArray(std::array<uint8_t, ArraySize> const & array, 
                                uint32_t & index); 

   // Note, not valid c++ since the size of the array (N) isn't 
   // specified. This has been omitted for simplicity. 
   void process(std::array<uin8_t, N> const & array)
   {
      auto currentIndex(u0);

      // Assumes this class has a specialization 
      // for getValueFromArray for all of the types in Types. 

      // This code doesn't work because there is no sequence point 
      // between each call to getValueFromArray, so the 
      // currentIndex can be incremented in a non-deterministic way. 
      func(this->getValueFromArray<Types>(array, currentIndex)...);
   }
};

I was able to work around this problem by introducing a new class:

template <typename T, std::size_t position>
struct Param
{
   using type = T;
   static constexpr std::size_t offset = position;
};

This way, instead of maintaining currentIndex at runtime, I can specify the offset of each argument in code, like this:

GENERIC_CLASS<Param<uint32_t, 0>, Param<uint8_t, 4>, Param<uint16_t, 5>>

The above is potentially error prone, as the offsets could be wrong. Is there some way to generate my sequence of Params from a parameter pack of types by accumulating sizes?

Alternatively, is there some workaround for the sequence point problem that I've mentioned above?

Could you elaborate? Using `c++11` I don't have access to `std::apply`, and trying to build the `std::tuple` from the `std::array` I would still encounter the sequencing problem. — Joshua Ryan, Jun 27 '18 at 14:13
Good news is I think it can be done. Bad news is it will probably require recursion and may not be suited for embedded systems — bartop, Jun 27 '18 at 14:46

max66 · Answer 1 · 2018-06-27T18:06:22.947

I propose the following solution

#include <array>
#include <tuple>
#include <iostream>

template <typename ... Ts>
class process
 {
   private:

      template <typename T>
      static T getVal (std::uint8_t const * a)
       {
         T ret { *a++ };

         for ( auto i = 1U ; i < sizeof(T) ; ++i )
          {
            ret <<= 8;
            ret  += *a++;
          }

         return ret;
       }

      static std::size_t postIncr (std::size_t & pos, std::size_t add)
       {
         std::size_t ret { pos };

         pos += add;

         return ret;
       }

   public: 
      template <std::size_t N>
      static std::tuple<Ts...> func (std::array<std::uint8_t, N> const & a)
       {
         std::size_t  pos { 0U };

         return { getVal<Ts>(a.data()+postIncr(pos, sizeof(Ts)))... };
       }
 };

int main ()
 { 
   std::array<std::uint8_t, 7U>
      bytes{{0x01U, 0x02U, 0x03U, 0x04U, 0x10U, 0x20U, 0x30U}};

   auto tpl
    { process<std::uint32_t, std::uint8_t, std::uint16_t>::func(bytes) };

   std::cout << "- 0x" << std::hex << std::get<0>(tpl) << std::endl;
   std::cout << "- 0x" << int(std::get<1>(tpl)) << std::endl;
   std::cout << "- 0x" << std::get<2>(tpl) << std::endl;
 }

If you get (as I get) an annoying warning from ret <<= 8; from getVal() (when T is std::uint8_t; warning "shift count >= width of type") you can develop a second getVal for std::uint8_t (without loop and without shift), enabling it only when sizeof(T) == 1 and enabling the first one only when sizeof(T) > 1.

Otherwise you can substitute ret <<= 8; with a couple of ret <<= 4;.

Your remark about endianess applies here too (it applies in fact to OP's post). — Jarod42, Jun 27 '18 at 17:52
@Jarod42 - I don't know... maybe, but in a different way. I mean: I interpret the question ("Where 0x01020304 is my uin32_t") in the sense that the order of the bytes in the array is starting from the most significant byte; so `memcpy()` should works only in one type of model (little endian? I ever make confusion); setting one byte at a time, and shifting, should works with both models. But I'm not sure to have understand the OP requirements. — max66, Jun 27 '18 at 18:05
@Jarod - I was wrong: the `memcy()` method should work (if I understand correctly the OP desiderata) only with big endian models (I **ever** make confusion). I've tried your solution in my Linux intel (little endian) netbook and, for first element, print "0x4030201"; my solution print "0x1020304". — max66, Jun 27 '18 at 18:35
One thing that I like better about @max66 's answer is that I intend to have different overloads of a function like `getVal` that supports non-packed types. For example, I might have a class that is a `uint8` and a `uint32`, so the specialization of `getVal` for that class would include one call to `getVal` for a `uint8` and one call for the `uint32`. — Joshua Ryan, Jun 27 '18 at 20:09

Jarod42 · Accepted Answer · 2018-06-27T16:10:50.073

1

It seems you want something like:

template <typename ...Ts>
void f(Ts... args)
{
    const int dummy[] = {0, ((std::cout << std::hex << args << std::endl), 0)...};
    static_cast<void>(dummy); // Avoid warning for unused variable
}

template <std::size_t N, typename Tuple, std::size_t ...Is>
void process(std::array<std::uint8_t, N> const& array, Tuple tuple, std::index_sequence<Is...>)
{
    int i = 0;
    const int dummy[] = {((memcpy(&std::get<Is>(tuple), array.data() + i, sizeof(std::tuple_element_t<Is, Tuple>)), i += sizeof(std::tuple_element_t<Is, Tuple>)), 0)...};
    static_cast<void>(dummy); // Avoid warning for unused variable
    f(std::get<Is>(tuple)...);
}

template <std::size_t N, typename ... Ts>
void process(std::array<std::uint8_t, N> const& array, std::tuple<Ts...> tuple)
{
    process(array, tuple, std::index_sequence_for<Ts...>{});
}

index_sequence can be implemented in C++11.

Demo

edited Jun 27 '18 at 16:10

answered Jun 27 '18 at 15:18

Jarod42

203,559
14
181
302

I think this is very close to what I'm trying to do. I'm going to try to find a C++11 implementation for `std::index_sequence` when I have some time, and I'll see if I can make this solution work for me. Thanks for the answer! – Joshua Ryan Jun 27 '18 at 15:30
Nice but... are we sure that is little endian / big endian safe ? – max66 Jun 27 '18 at 16:51
1

A little suggestion: pass `tuple` as reference in both `process()` function. – max66 Jun 27 '18 at 18:30

rmawatson · Answer 3 · 2018-06-27T15:30:28.103

How about something like this...

#include <iostream>
#include <type_traits>
#include <cstring>
#include <typeinfo>

using namespace std;

template <size_t O,size_t I,size_t C,typename...>
struct byte_offset_impl;

template <size_t O,size_t I,size_t C,typename T,typename... Ts>
struct byte_offset_impl< O, I, C,T,Ts...> : 
    std::conditional<I == C,
      std::integral_constant<size_t,O>,
      byte_offset_impl<O + sizeof(T),I,C+1,Ts...>    
    >::type
{};


template <typename T>
struct type_c 
{ using type = T; };

template <size_t I,size_t C,typename...>
struct type_at_impl;

template <size_t I,size_t C,typename T,typename... Ts>
struct type_at_impl<I, C,T,Ts...> : 
    std::conditional<I == C,
      type_c<T>,
      type_at_impl<I,C+1,Ts...>  
    >::type
{};


template <size_t I,typename... Ts>
constexpr size_t byte_offset = byte_offset_impl<0,I,0,Ts...>::value;

template <size_t I,typename... Ts>
using type_at = typename type_at_impl<I,0,Ts...>::type;

template <typename...Ts>
struct generic_class
{
    generic_class(char* byteptr) : byteptr_(byteptr)
    {};


    template <size_t I>
    auto get() -> type_at<I, Ts...>& // needed for c++11
    {
        using result_type = type_at<I, Ts...>;
        return *reinterpret_cast<result_type*>(&byteptr_[byte_offset<I, Ts...>]);

    }

    template <size_t I>
    auto get_v() -> type_at<I, Ts...> // needed for c++11
    {
        using result_type = type_at<I, Ts...>;
        result_type result;

        std::memcpy(&result, &byteptr_[byte_offset<I, Ts...>], sizeof(result_type));

        return result;
    }
    char* byteptr_;
};


int main() {

    char bytes[sizeof(uint32_t) + sizeof(uint8_t) + sizeof(uint16_t)];

    uint32_t u32 = 1561;
    uint8_t u8 = 22;
    uint16_t u16 = 99;

    char* bytesp = bytes;
    std::memcpy(bytesp, &u32, sizeof(uint32_t)); bytesp += sizeof(uint32_t);
    std::memcpy(bytesp, &u8, sizeof(uint8_t)); bytesp += sizeof(uint8_t);
    std::memcpy(bytesp, &u16, sizeof(uint16_t));

    generic_class<uint32_t, uint8_t, uint16_t> gen(bytes);
    std::cout << (uint32_t)gen.get<0>() << std::endl;
    std::cout << (uint32_t)gen.get<1>() << std::endl;
    std::cout << (uint32_t)gen.get<2>() << std::endl;

    std::cout << (uint32_t)gen.get_v<0>() << std::endl;
    std::cout << (uint32_t)gen.get_v<1>() << std::endl;
    std::cout << (uint32_t)gen.get_v<2>() << std::endl;
}

Demo

Depending on your platform you may find the get<I> implementation not suitable, as access may be unaligned. the get_v<I> alternative will copy the data out of your array into an element of the correct type, with the added benefit that it will only copy if the data is accessed.

score -1 · Answer 4 · answered Jun 27 '18 at 15:19

Each class needs an implementation of as_tuple, returning a tie of the members.

class Object
{
    // Private members
    uint32_t val1;
    uint8_t val2;
    uint16_t val3;
public:
    auto as_tuple()
    {
        return std::tie(val1, val2, val3);
    }
    // Use Object
    friend std::ostream& operator<<(std::ostream& os, Object& o)
    {
        return os << o.val1 << ' ' << int(o.val2) << ' ' << o.val3;
    }
};

You can then copy into each member

namespace detail
{
    // To verify the size of the input array matches the sum of the member's sizes
    template <std::size_t...> struct sum;
    template <> struct sum<> { static constexpr std::size_t value = 0; };
    template <std::size_t I, std::size_t... Is> struct sum<I, Is...> { static constexpr std::size_t value = I + sum<Is...>::value; };

    template <std::size_t... Is> constexpr std::size_t sum_v = sum<Is...>::value;

    // Remove the front element of a tuple
    template<typename T, typename Seq>
    struct tuple_cdr_impl;

    template<typename T, std::size_t I0, std::size_t... I>
    struct tuple_cdr_impl<T, std::index_sequence<I0, I...>>
    {
        using type = std::tuple<typename std::tuple_element<I, T>::type...>;
    };

    template<typename T>
    struct tuple_cdr
    : tuple_cdr_impl<T, std::make_index_sequence<std::tuple_size<T>::value>>
    { };

    template<typename T>
    using tuple_cdr_t = typename tuple_cdr<T>::type;

    template<typename T, std::size_t I0, std::size_t... I>
    tuple_cdr_t<typename std::remove_reference_t<T>>
    cdr_impl(T&& t, std::index_sequence<I0, I...>)
    {
        return std::tie(std::get<I>(t)...);
    }

    template<typename T>
    tuple_cdr_t<typename std::remove_reference_t<T>>
    cdr(T&& t)
    {
        return cdr_impl(std::forward<T>(t),
                        std::make_index_sequence<std::tuple_size<typename std::remove_reference_t<T>>::value>{});
    }

    // copy from bytes into a reference
    void fromBytesImpl(const unsigned char *, std::tuple<>)
    {
    }

    template <typename T, typename... Ts>
    void fromBytesImpl(const unsigned char * arr, std::tuple<T&, Ts&...> tup)
    {
        auto dest = reinterpret_cast<unsigned char *>(&std::get<0>(tup));
        std::copy_n(arr, sizeof(T), dest);
        fromBytesImpl(arr + sizeof(T), cdr(tup));
    }
}

// Bringing it all together
template <typename... Ts>
void fromBytes(const std::array<unsigned char, detail::sum_v<sizeof(Ts)...>> & arr, std::tuple<Ts&...> tup)
{
    detail::fromBytesImpl(arr.data(), tup);
}

This is then simple to use

int main()
{
    Object obj;
    std::array<unsigned char, 7> bytes = {0x01, 0x02, 0x03, 0x04, 0x10, 0x20, 0x30};
    fromBytes(bytes, obj.as_tuple());

    std::cout << std::hex << obj << std::endl;
}

See it live

How Can I Build A Sequence Of Structs With Integer Members of Increasing Value at Compile Time in C++11

4 Answers4