MsgPack Woes - Destructor, pack_raw_body, Etc

Question

I am using MsgPack as part of a custom back-end I am creating for the SOCI C++ database access API. Initially, some of my SOCI classes had MsgPack::sbuffer objects as member variables, but I ran into some problems in the destructor of my SOCI object - I believe the problem relates to the way that SOCI reference counts some of its objects and the memory that underlies the MsgPack object being freed multiple times.

In an attempt to remedy this problem, I decide to replace the MsgPack::sbuffer member variables with std::vector<char> member variables, and use these to fill the MsgPack::sbuffers using the pack_raw_body method. Unfortunately, I'm having trouble with this as well.

Please consider the following (pseudo-)code snippet...

msgpack::sbuffer buf1;
msgpack::packer<msgpack::sbuffer> bufPkr1(&buf1);
bufPkr1.pack_array(num);
for (int ndx = 0; ndx < num; ++ndx) {
  bufPkr1.pack_array(3);
  bufPkr1.pack(std::string("foo"));
  bufPkr1.pack(std::string("bar"));
  bufPrk1.pack(221);
}

std::vector<char> chrVct = std::vector<char>(buf1.size(), *buf1.data());

msgpack::unpacked unPkd1;
msgpack::unpack(&unPkd1, buf1.data(), buf1.size());

msgpack::object toStr1 = unPkd1.get();
std::cout << "MsgPack1: " << toStr1 << std::endl;

msgpack::sbuffer buf2;
msgpack::packer<msgpack::sbuffer> bufPkr2(&buf2);
bufPkr1.pack_raw(chrVct.size());
bufPkr1.pack_raw_body(chrVct.data(), chrVct.size());

msgpack::unpacked unPkd2;
msgpack::unpack(&unPkd2, buf2.data(), buf2.size());

msgpack::object toStr2 = unPkd2.get();
std::cout << "MsgPack2: " << toStr2 << std::endl;

Output...

MsgPack1: [["foo", "bar", 221], ["foo", "bar", 221], ["foo", "bar", 221],..., ["foo", "bar", 221]]
MsgPack2: ""

In general, I'm just having trouble figuring out how to work with the MsgPack objects and am finding the documentation/examples a little sparse. Any help that people can offer would be much appreciated!

score 0 · Accepted Answer · answered May 12 '14 at 20:14

One of the quirks of the MsgPack API seems to have to do with the lifetime of the various MsgPack objects, as well as the objects/integral data types that are being packed into the MsgPack objects.

The above code probably would have worked better if I had followed this idiom...

msgpack::sbuffer buf1;
msgpack::packer<msgpack::sbuffer> bufPkr1(&buf1);
bufPkr1.pack_array(num);
for (int ndx = 0; ndx < num; ++ndx) {
  bufPkr1.pack_array(3);
  bufPkr1.pack(std::string("foo"));
  bufPkr1.pack(std::string("bar"));
  bufPrk1.pack(221);
}

msgpack::unpacked unPkd1;
msgpack::unpack(&unPkd1, buf1.data(), buf1.size());
const msgpack::object obj1 = respUnPk.get();

msgpack::sbuffer buf2;
msgpack::packer<msgpack::sbuffer> bufPkr2(&buf2);
bufPkr2.pack(obj1);

score 0 · Answer 2 · answered Aug 08 '14 at 08:12

If I understand your question correctly, you want to pack the data in buf1 as a raw binary image. Then you want to unpack it.

The first problem is the usage of the std::vector's constructor. See the comment P1 in the following code:

#include <msgpack.hpp>
#include <iostream>
#include <iomanip>
#include <algorithm>

int main() {
    int const num = 5;
    msgpack::sbuffer buf1;
    msgpack::packer<msgpack::sbuffer> bufPkr1(&buf1);
    bufPkr1.pack_array(num);
    for (int ndx = 0; ndx < num; ++ndx) {
        bufPkr1.pack_array(3);
        bufPkr1.pack(std::string("foo"));
        bufPkr1.pack(std::string("bar"));
        bufPkr1.pack(221);
    }

// P1
//  std::vector<char> chrVct = std::vector<char>(buf1.size(), *buf1.data());
    std::vector<char> chrVct = std::vector<char>(buf1.data(), static_cast<char*>(buf1.data()) + buf1.size());

    msgpack::unpacked unPkd1;
    msgpack::unpack(&unPkd1, buf1.data(), buf1.size());

    msgpack::object toStr1 = unPkd1.get();
    std::cout << "MsgPack1: " << toStr1 << std::endl;

    msgpack::sbuffer buf2;
    msgpack::packer<msgpack::sbuffer> bufPkr2(&buf2);

// P2
//  bufPkr1.pack_raw(chrVct.size());
//  bufPkr1.pack_raw_body(chrVct.data(), chrVct.size());
    bufPkr2.pack_raw(chrVct.size());
    bufPkr2.pack_raw_body(chrVct.data(), chrVct.size());

    msgpack::unpacked unPkd2;
    msgpack::unpack(&unPkd2, buf2.data(), buf2.size());

    msgpack::object toStr2 = unPkd2.get();
    std::cout << "MsgPack2: " << toStr2.via.raw.size << std::endl;
    std::for_each(toStr2.via.raw.ptr, toStr2.via.raw.ptr+toStr2.via.raw.size,
            [](char e) {
                std::cout << std::hex << std::setw(2) << std::setfill('0') << (static_cast<int>(e) & 0xff) << ' ';
            });
    std::cout << std::endl;
}

The std::vector's constructor in the original code is fill all the vector member to the same value that is passed as the second parameter.

That is the (2) of the following documentation:

http://www.cplusplus.com/reference/vector/vector/vector/

I believe that the third constructor, range version, should be used on this situation. So I replaced the original code with modified code.

The second problem is that bufPkr2 is never used. I guess that it is just typo. I commented out the original code and replaced it.

As the result of those modification, I got the following result:

MsgPack1: [["foo", "bar", 221], ["foo", "bar", 221], ["foo", "bar", 221], ["foo", "bar", 221], ["foo", "bar", 221]]
MsgPack2: 56
95 93 a3 66 6f 6f a3 62 61 72 cc dd 93 a3 66 6f 6f a3 62 61 72 cc dd 93 a3 66 6f 6f a3 62 61 72 cc dd 93 a3 66 6f 6f a3 62 61 72 cc dd 93 a3 66 6f 6f a3 62 61 72 cc dd

MsgPack Woes - Destructor, pack_raw_body, Etc

2 Answers2