How to create an array of callables based on a std::variant

Question

How can I create an array of functions that based off a std::variant which composed of a few message types, where they are to be decoded from io bytes, so that I can quickly access the right functions based on a mid field from the bytes.

struct S1 { constexpr static int MID = 1; };
struct S2 { constexpr static int MID = 2; };
struct S3 { constexpr static int MID = 3; };
struct S4 { constexpr static int MID = 4; };
struct S5 { constexpr static int MID = 5; };

typedef std::variant<S1,S2,S3,S4,S5> MyVariant;


template<class M> void decode(M* output, const std::vector<char>& input)
{
    // generic decoding for everything else
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

template<> void decode(S3* output, const std::vector<char>& input)
{
    // speical decoding for S3
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

template<> void decode(S5* output, const std::vector<char>& input)
{
    // speical decoding for S5
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

I tried this:

using Decode_func = void(void*, const std::vector<char>& input);
constexpr std::array<Decode_func*, std::variant_size_v<MyVariant>> funcs = {};

But this doesn't work because the first parameter of Decode_func needs to match the exact parameter but in function template, the actual type is unknown. Also how can I fill the funcs array in compile time?

What I wanna achieve in the end is like this:

std::vector<char> buffer = read(...);
int mid = getMid(buffer);

std::variant_alternative_t<mid, MyVariant> msg;
Decode_func *decoder = funcs[mid-1];  // how can I build and fill funcs from decode functions at compile time

decoder(&msg, buffer);
MyVariant v(msg);
// pass v for processing

Maybe an abstract factory could help? But for me, it is not really sure what you want to achieve. — A M, Feb 25 '23 at 09:00
`std::variant_alternative_t msg;` won't work with an int read at runtime. — joergbrech, Feb 25 '23 at 09:16

Ted Lyngmo · Answer 1 · 2023-02-25T14:03:30.263

You need some way to get from the raw bytes in input to a type. The index template parameter to std::variant_alternative must be a compile time constant and the MID bytes inside input are not compile time constants - so you can't use that.

MyVariant decode(const std::vector<uint8_t>& input) {
    if(input.size() < 4) throw std::runtime_error("nope");

    // extract `mid` (use whatever way you use already):
    uint32_t mid = (uint32_t(input[0]) << 24) +
                   (uint32_t(input[1]) << 16) +
                   (uint32_t(input[2]) << 8) + input[3];

    switch(mid) { // lookup the right function to use for decoding
    case S3::MID: return decode<S3>(input);
    case S5::MID: return decode<S5>(input);
    default:        
        MyVariant rv;
        //generic decoding
        return rv;
    }
}

Where the decode function template and specializations could be:

template<class S> MyVariant decode(const std::vector<uint8_t>&);

template <>
MyVariant decode<S3>(const std::vector<uint8_t>& input) {
    S3 rv; // special decoding for S3
    // ...
    return rv;
}

template <>
MyVariant decode<S5>(const std::vector<uint8_t>& input) {
    S5 rv; // special decoding for S5
    // ...
    return rv;
}

A second option could be to put the decoding into the constructor of the actual types:

struct generic_decoder { 
    generic_decoder(const std::vector<uint8_t>& input) {} 
};

struct S1 : generic_decoder { constexpr static uint32_t MID = 1; };
struct S2 : generic_decoder { constexpr static uint32_t MID = 2; };
struct S3 {
    constexpr static uint32_t MID = 3;
    S3(const std::vector<uint8_t>& input) {
        // special decoding for S3
    }
};
struct S4 : generic_decoder { constexpr static uint32_t MID = 4; };
struct S5 {
    constexpr static uint32_t MID = 5;
    S5(const std::vector<uint8_t>& input) {
        // special decoding for S5
    }
};

This would let the lookup and creation part be pretty clean:

MyVariant decode(const std::vector<uint8_t>& input) {
    if (input.size() < 4) throw std::runtime_error("nope");

    uint32_t mid = (uint32_t(input[0]) << 24) + (uint32_t(input[1]) << 16) +
                   (uint32_t(input[2]) << 8) + input[3];

    switch(mid) {
        case S1::MID: return S1(input);
        case S2::MID: return S2(input);
        case S3::MID: return S3(input);
        case S4::MID: return S4(input);
        case S5::MID: return S5(input);
        default: throw std::runtime_error("invalid MID in input");
    }
}

A third, more expensive, version could be to first create the MyVariant instance and then use std::visit to decode. It works, but it's unnecessary since you already know the type you put in the variant. Anyway, it could look like this:

struct generic_decoder { void decode(const std::vector<uint8_t>& input) {} };

struct S1 : generic_decoder { constexpr static uint32_t MID = 1; };
struct S2 : generic_decoder { constexpr static uint32_t MID = 2; };
struct S3 {
    constexpr static uint32_t MID = 3;
    void decode(const std::vector<uint8_t>& input) {
        // special decoding for S3
    }
};
struct S4 : generic_decoder { constexpr static uint32_t MID = 4; };
struct S5 {
    constexpr static uint32_t MID = 5;
    void decode(const std::vector<uint8_t>& input) {
        // special decoding for S5
    }
};

using MyVariant = std::variant<S1, S2, S3, S4, S5>;

And create an array of MyVariant with default constructed in instances of your types. You then copy the correct variant by indexing and then std::visit to decode:

MyVariant decode(const std::vector<uint8_t>& input) {
    static const std::array<MyVariant, 5> vs{S1{}, S2{}, S3{}, S4{}, S5{}};

    if (input.size() < 4) throw std::runtime_error("nope");

    uint32_t mid = (uint32_t(input[0]) << 24) + (uint32_t(input[1]) << 16) +
                   (uint32_t(input[2]) << 8) + input[3];

    auto rv = vs.at(mid - 1); // throws if `mid` is invalid
    std::visit([&](auto&& var) { var.decode(input); }, rv);
    return rv;
}

joergbrech · Answer 2 · 2023-02-25T11:33:21.133

How about this? You create an array of Callables to create the correct MyVariant variant, based on the integer you read at runtime. Then you add an additional specialization for decode that accepts MyVariant and dispatches to the correct decode specialization for the held variant via std::visit:

#include <variant>
#include <array>
#include <functional>
#include <iostream>

struct S1 { constexpr static int MID = 1; };
struct S2 { constexpr static int MID = 2; };
struct S3 { constexpr static int MID = 3; };
struct S4 { constexpr static int MID = 4; };
struct S5 { constexpr static int MID = 5; };

typedef std::variant<S1,S2,S3,S4,S5> MyVariant;

template <class M>
void decode(M* output, const std::vector<char>& input) {
    // generic decoding for everything else
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

template <>
void decode(S3* output, const std::vector<char>& input) {
    // speical decoding for S3
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

template <>
void decode(S5* output, const std::vector<char>& input) {
    // speical decoding for S5
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

template <>
void decode(MyVariant* output, const std::vector<char>& input)
{
    std::visit([&input](auto out) { decode(&out, input); }, *output);
}

std::array<std::function<MyVariant()>, 5> variantfactory = {
    []{ return S1{}; },
    []{ return S2{}; },
    []{ return S3{}; },
    []{ return S4{}; },
    []{ return S5{}; },
};

int main() {
    // read from buffer
    std::vector<char> buffer;
    int mid = 3;

    auto msg = variantfactory[mid-1]();
    decode(&msg, buffer);
    // pass msg for processing
}

23 void decode(M*, const std::vector<char>&) [with M = S3]

https://godbolt.org/z/bM7YGKheW

Marcus Müller · Answer 3 · 2023-02-25T08:45:02.823

This reads a lot like you're trying to reinvent virtual inheritance, a feature that C++ already has with classes; no need to use a std::variant.

But assuming you have good reason to use std::variant instead, your question becomes basically "How do I use the currently active type of a variant as typename, in order to resolve functions", and the answer is: you'll need to write a visitor that, at runtime, makes that type distinction.

std::visit allows you to do that. Cppreference's page on it has a workable example, something that's reworkable to

#include <iostream>
#include <variant>
#include <vector>

struct S1 {
    constexpr static int MID = 1;
};
struct S2 {
    constexpr static int MID = 2;
};
struct S3 {
    constexpr static int MID = 3;
};
struct S4 {
    constexpr static int MID = 4;
};
struct S5 {
    constexpr static int MID = 5;
};

using MyVariant = std::variant<S1, S2, S3, S4, S5>;

template <class M>
void decode(M* output, const std::vector<char>& input) {
    // generic decoding for everything else
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

template <>
void decode(S3* output, const std::vector<char>& input) {
    // speical decoding for S3
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

template <>
void decode(S5* output, const std::vector<char>& input) {
    // speical decoding for S5
    std::cout << __LINE__ << " " << __PRETTY_FUNCTION__ << std::endl;
}

int main() {
    MyVariant s1var;
    S3 s3;
    S5 s5;
    MyVariant s3var(s3);
    MyVariant s5var(s5);
    std::vector<char> inp;
    for (const auto& var : {s1var, s3var, s5var}) {
        std::visit([inp](const auto& out) { decode(&out, inp); }, var);
    }
}

(run in Godbolt Compiler Explorer)

Thanks it's a little different, I don't have a variant, what I need is first decode the struct from the buffer, then create a `std::variant`. I'm stuck on the 1st step now. — fluter, Feb 25 '23 at 08:37
but then, why are you type-erasing in your `Decode_func`? If you did *not* do that conversion of the specific-type pointer to `void*`, normal argument type-dependent function lookup just works? So, call `decode` with your object directly? — Marcus Müller, Feb 25 '23 at 08:48
I don't have the structs or the variant in the beginning, I need to decode it first then create a variant, I just updated my question on what I am looking for. — fluter, Feb 25 '23 at 09:01

How to create an array of callables based on a std::variant

3 Answers3