0

I'm trying to use Cap’n Proto in existing project consisting of client and server communicating over UDS. I don't have the resources (and I doubt it would be accepted) to redo all client-server RPC, but I wanted to benefit from Cap’n Proto serialization mechanisms. Unfortunately, it seems to me it's impossible.

The biggest problem is server side, which is single threaded (and it will remain so, if there aren't any serious arguments for multithreading) and uses it's own poll based loop. All events are read partially, server can't block waiting for any event to be fully read - and this is where I am stuck. We have our own protocol and classes which wrap the message, which can consume bytes from file descriptor and notify, when the event is fully read, so the server can process it. I think I've analysed most of Cap’n Proto interfaces (serialization, async serialization) and it seems, that it can't be used the same way without any modifications.

I really hope that I've missed something. Did I?

zoska
  • 1,684
  • 11
  • 23

1 Answers1

1

There are two ways you can solve this:

  • Hard way: You can attempt to integrate with the KJ async I/O framework (used by Cap'n Proto). The KJ event loop can actually integrate with other event loops and run on top of them -- but it's tricky. For example, node-capnp includes code to integrate the KJ event loop with libuv, as seen in the first part of this source file. Once you have the necessary glue, you can write KJ-style async code that uses the interfaces in capnp/serialize-async.h.
  • Easy way: Instead of trying to integrate KJ, you can write simple code using your event infrastructure which reads data from the file descriptor directly and then uses capnp::expectedSizeInWordsFromPrefix() (from capnp/serialize.h) to figure out if it has received the whole message yet. If that function returns a number greater than what you already have, then you don't have the full message and have to keep waiting. Once you have the full message, you can then use capnp::FlatArrayMessageReader to parse it.
Kenton Varda
  • 41,353
  • 8
  • 121
  • 105
  • Thank you. I really appreciate code author, who is active on Stack Overflow :) – zoska Oct 30 '17 at 15:50
  • Is expectedSizeInWordsFromPrefix working for packed messages too? I always get huge numbers... – hunyadym Nov 22 '17 at 22:22
  • @hunyadym No, it doesn't work for packed messages. Implementing something like this for packed messages will actually be rather difficult. You might be better off writing a length prefix explicitly. :/ I've filed an issue for this: https://github.com/capnproto/capnproto/issues/590 – Kenton Varda Nov 22 '17 at 23:51
  • Is there a way of knowing, what type of message is sent through the socket? Or could you suggest best practice for sending operation type first? – zoska Dec 08 '17 at 12:42
  • @zoska I recommend creating one master message which contains a `union` of all message types you want to support. Then, always send and receive instances of the master message. – Kenton Varda Dec 12 '17 at 17:41