how to put multiple protobufs on a wire

Question

We have a communication channel on which we send protobufs. In order to be able to send more than one type of protobuf, we double-serialise:

message Coprolite {
    enum Bezoar {
        KBezoarUndef = 0;
        KBezoarFriedEggs = 1;
        KBezoarHam = 2;
    }
    Bezoar payload_type = 1;
    bytes payload = 2;
}

If I have a FriedEggs protobuf, I serialise it, assign it to the payload of a Coprolite, set the payload_type to KBezoarFriedEggs, serialise the Coprolite, and send it on its way.

On receipt, I deserialise, check what I've got, and deserialise that.

This works on all of our platforms. I've not, however, found examples of others doing this this way (nor any other way, really). So this suggests I should ask for advice. Is there a better strategy or a reason that I should be wary of this?

score 2 · Accepted Answer · answered Mar 26 '20 at 12:38

If you want to prevent having to set payload_type you can use an oneof. Oneof's implicitly serialize the payload type by adding the tag number in front. Just like any other field serialized. So you have to write less administrative code.

There is however one advantage in your approach over using oneof's. Deepening on your programming language and how oneof's and byte arrays are implemented on your platform and in your protobuf library. An oneof implemented as an union might allocate memory the size of the largest nested message. So depending on your situation dynamically allocating the bytes array might use less memory when you send a lot of small message and only sometimes a big one.

score 2 · Answer 2 · answered Mar 26 '20 at 13:01

There are two common approaches here; the simplest (especially when the types are predictable in advance) is usually oneof:

message Coprolite {
    oneof payload_type {
         FriedEggs eggs = 1;
         Ham ham = 2;
    }
}

This acts like a discriminated union, where you can check the embedded type.

In some niche scenarios, you may prefer to use Any

how to put multiple protobufs on a wire

2 Answers2