Deserializing a heterogeneous map with MessagePack in C++

Question

I’m using MessagePack with C++ and I’m trying to deserialize the equivalent of this Python map:

{'metadata': {'date': '2014-06-25', 'user_id': 501},
 'values': [3.0, 4.0, 5.0],
 'version': 1}

The top-level object is a map with string keys, but the values are of completely different types. My code knows ahead of time what the structure of the object should be; I ought to be able to declare an integer and then tell my deserialization code, “The value of the version key is an integer, so put the value of that integer into this memory address.”

The problem is that I’m not even sure how to get to the point where my C++ code can treat this structure as a map. I would expect to do something like

msgpack::unpacker unpacker;
// ...copy the data into unpacker's buffer...

msgpack::unpacked message;
std::map<std::string, anything> output_map;

unpacker.next(&message);
msgpack::object obj = message.get();
obj.convert(&output_map);

int version_number = output_map.at("version");

Is there any possible type (anything) that would work here? The MessagePack documentation has only trivial examples, and this blog post is better but doesn’t cover this use case.

`boost::any` or `boost::variant` could do the job, but this problem is a bigger design question than you think, your map could have another map as one of its members, so you'll ultimately need a completely recursive map, which is more or less a tree. Most JSON parsing libraries end up creating their own custom type to store this kind of data — Drax, Jun 25 '14 at 15:27

score 1 · Answer 1 · answered Jun 28 '14 at 16:28

You can do that using boost::variant. To implement recursive structure, you can use oost::make_recursive_variant as follows:

typedef boost::make_recursive_variant<
    std::string,
    std::map<boost::recursive_variant_, boost::recursive_variant_>,
    std::vector<boost::recursive_variant_>,
    int,
    double
    >::type variant_t;

Here is a documentation: http://www.boost.org/doc/libs/1_55_0/doc/html/variant/tutorial.html#variant.tutorial.recursive.recursive-variant

You also need to write a converter that convert from msgpack::object to variant_t and vice versa as follows:

// Custom converter for variant_t
namespace msgpack {

// Convert from msgpacl::object to variant_t.
inline variant_t& operator>>(object const& o, variant_t& v) {
    switch(o.type) {
    case type::MAP:
        v = std::map<variant_t, variant_t>();
        o.convert(boost::get<std::map<variant_t, variant_t> >(&v));
        break;
    case type::ARRAY:
        v = std::vector<variant_t>();
        o.convert(boost::get<std::vector<variant_t> >(&v));
        break;
    case type::POSITIVE_INTEGER:
        v = int();
        o.convert(boost::get<int>(&v));
        break;
    case type::DOUBLE:
        v = double();
        o.convert(boost::get<double>(&v));
        break;
    case type::RAW:
        v = std::string();
        o.convert(boost::get<std::string>(&v));
        break;
    default:
        break;
    }
    return v;
}


// Convert from variant_t to msgpacl::object.
template <typename Stream>
struct packer_imp:boost::static_visitor<void> {
    template <typename T>
    void operator()(T const& value) const {
        o_.pack(value);
    }
    packer_imp(packer<Stream>& o):o_(o) {}
    packer<Stream>& o_;
};

template <typename Stream>
inline packer<Stream>& operator<< (packer<Stream>& o, const variant_t& v)
{
    boost::apply_visitor(packer_imp<Stream>(o), v);
    return o;
}

} // namespace msgpack

You can get a full example code from gist: https://gist.github.com/redboltz/672c5af16b2907488977 I've used the C++11 feature in the example, so you need to add -std=c++11 option.

score 0 · Accepted Answer · edited May 23 '17 at 11:53

0

I ended up punting on this and writing my own text-based serialization format. It’s not generally useful like MessagePack is, but it allows me to paper over these static-typing woes.

edited May 23 '17 at 11:53

Community

1
1

answered Aug 07 '14 at 20:58

bdesham

15,430
13
79
123

Deserializing a heterogeneous map with MessagePack in C++

2 Answers2

Linked