0

I'm working with Cap'n'Proto and my understanding is there is no need to do serialization as it's already being done. So my question is, how would I access the serialized data and get it's size so that I can pass it in as a byte array to another library.

// person.capnp
struct Person {
    name @0 :Text;
    age @1 :Int16;
}

// ...
::capnp::MallocMessageBuilder message;

Person::Builder person = message.initRoot<Person>();
person.setName("me");
person.setAge(20);

// at this point, how do I get some sort of handle to 
// the serialized data of 'person' as well as it's size?

I've seen the writePackedMessageToFd(fd, message); call, but didn't quite understand what was being passed and couldn't find any API docs on it. I also wasn't trying to write to a file descriptor as I need the serialized data returned as const void*.

Looking in Capnproto's message.h file is this function which is in the base class for MallocMessageBuilder which says it gets the raw data making up the message.

kj::ArrayPtr<const kj::ArrayPtr<const word>> getSegmentsForOutput();
// Get the raw data that makes up the message.

But even then, Im' not sure how to get it as const void*.

Thoughts?

Ender
  • 1,652
  • 2
  • 25
  • 50

2 Answers2

0
::capnp::MallocMessageBuilder message;

is your binary message, and its size is

message.sizeInWords()

(size in bytes divided by 8).

bipll
  • 11,747
  • 1
  • 18
  • 32
  • I gave that a try and things blew up. Trying to figure out what that's returning. Will keep working with it. – Ender Jun 23 '21 at 22:26
0

This appears to be whats needed.

// ...
::capnp::MallocMessageBuilder message;

Person::Builder person = message.initRoot<Person>();
person.setName("me");
person.setAge(20);

kj::Array<capnp::word> dataArr = capnp::messageToFlatArray(message);
kj::ArrayPtr<kj::byte> bytes = dataArr.asBytes();

std::string data(bytes.begin(), bytes.end());

const void* dataPtr = data.c_str();

At this point, I have a const void* dataPtr and size using data.size().

Ender
  • 1,652
  • 2
  • 25
  • 50
  • Apparently the issue is that the over-the-wire representation includes multiple "segments" and you can't just concatenate them together - there's some metadata at the beginning. See [here](https://github.com/capnproto/capnproto/blob/6b5bcc2c6e954bc6e167ac581eb628e5a462a469/c%2B%2B/src/capnp/serialize.h#L216) where `messageToFlatArray` gets those segments (even if there's only one, I guess) and then calls [here](https://github.com/capnproto/capnproto/blob/6b5bcc2c6e954bc6e167ac581eb628e5a462a469/c%2B%2B/src/capnp/serialize.c%2B%2B#L118) to glue them together into the buffer. – davidbak Jun 24 '21 at 17:02
  • (Which I guess means an extra copy is unavoidable, unless you just duplicate the functionality of [`writeMessage`](https://github.com/capnproto/capnproto/blob/6b5bcc2c6e954bc6e167ac581eb628e5a462a469/c%2B%2B/src/capnp/serialize.c%2B%2B#L272) to copy it where you really want it.) – davidbak Jun 24 '21 at 17:03
  • Thanks for that. I'll have to try and understand if I'm getting all segments or if I need to do something more since I'm already using ```messageToFlatArray```. I'll look into ```writeMessage``` as well. I just need to hand off a ```const void*``` to another library. – Ender Jun 24 '21 at 23:33
  • `messageToFlatArray` itself copies _all_ segments to a buffer, hands you back that buffer. But it sticks the # of segments - 1 at the front of that buffer first (I guess so the deserializer knows how many it needs to deserialize). – davidbak Jun 24 '21 at 23:56
  • By the way, unless you're desperate to have the trailing `\0` character that the string adds to the end of the buffer you get back with `c_str()` - consider _not_ copying the buffer out of the `kj::Array` `dataArr` and instead just using it directly with `kj::Array::asBytes()` or `kj::Array::asChars()` or even, if you want to take ownership of the buffer out of the `kj::Array` use `kj::Array::releaseAsBytes/Chars()`. Save a buffer copy each time, if that's important to you. – davidbak Jun 25 '21 at 00:01