How to deserialize a std::string to a concrete message in Cap'n Proto C++?

Question

I need to decode a message from a std::string object.

I can see how to do this in Java by using a ByteBuffer, but I failed to find a simple C++ equivalent.

More precisely, how to implement this?

const std::string serialized_data = ...; // Source
SomeDataType data = ???; // How to convert from serialized_data to data?

*I can see how to do this in Java by using a ByteBuffer,* -- Without mentioning Java, what exactly are you trying to accomplish with `std::string`? — PaulMcKenzie, Apr 13 '22 at 13:45
You also have c++ samples here, https://github.com/capnproto/capnproto/blob/master/c%2B%2B/samples/addressbook.c%2B%2B — Captain Giraffe, Apr 13 '22 at 13:47
std::string is actually similar thig to java's ByteBuffer. Java's string are constant, C++ standard library - are mutable. — Victor Gubin, Apr 13 '22 at 13:48
@PaulMcKenzie I am writing a JNI library, the path is Java message object -> java byte array -> C++ char*/string -> C++ message object. — user2595776, Apr 13 '22 at 13:48
@CaptainGiraffe Yes, I saw that example. But I don't have socket, or file, or fd, only `std::string`. I tried to approach with `kj::ArrayInputStream` but it felt too clumsy. — user2595776, Apr 13 '22 at 13:51
@user2595776 Did you see that `std::string` has a `c_str()` and `data()` member functions? Why not utilize those functions? They exist because it is well-known that `std::string` may interact with regular C-style functions that handle "regular" character buffers. — PaulMcKenzie, Apr 13 '22 at 13:52
@PaulMcKenzie I know but there doesn't seem to be a simple direct conversion from `char*` to `MessageReader`. I only found a way to construct `InputStreamMessageReader` from `ArrayInputStream`, which itself is from an `ArrayPtr`. Do you have other suggestions? — user2595776, Apr 13 '22 at 13:57
I'm just skimming the docs, but I spotted `capnp::FlatArrayMessageReader`, given as an example of something usable with `mmap()`, your `std::string` contents aren't very different. — Hasturkun, Apr 13 '22 at 14:01

score 2 · Accepted Answer · answered Apr 14 '22 at 00:01

You can write code like this:

std::string serialized_data = ...;
kj::ArrayPtr<const capnp::word> words(
    reinterpret_cast<const capnp::word*>(serialized_data.begin()),
    serialized_data.size() / sizeof(capnp::word));
capnp::FlatAraryMessageReader reader(words);

Note that this assumes that the backing buffer of an std::string is always word-aligned. I think this is true, but I am not sure. If it is not aligned, you will get exceptions saying so. In this case it may be necessary to copy the data into an aligned buffer:

auto words = kj::heapArray<capnp::word>(
    serialized_data.size() / sizeof(capnp::word));
memcpy(words.begin(), serialized_data.begin(), words.asBytes().size());
capnp::FlatAraryMessageReader reader(words);

Note that while std::string is technically able to store arbitrary bytes, it is not a particularly good data structure for this purpose, since it is optimized for text. For example, it may re-allocate and copy your buffer in order to add a NUL terminator to support .c_str(). I would recommend using a more data-oriented type for byte arrays.

score 0 · Answer 2 · answered Apr 14 '22 at 00:52

My own answer is very similar to Kenton's:

std::string serialized_data = ...;
kj::ArrayPtr<const kj::byte> arr(
  reinterpret_cast<const kj::byte*>(serialized_data.data()),
  serialized_data.size());
kj::ArrayInputStream stream(arr);
capnp::InputStreamMessageReader reader(stream);
auto data = reader.getRoot<SomeDataType>();

This is isomorphic to the Java version, and it has no alignment requirements.

But Kenton's solution is more straightforward than this. Therefore I will simplify my program further.

How to deserialize a std::string to a concrete message in Cap'n Proto C++?

2 Answers2