Is it best to have many field in protobuf message or nested messages?

Question

I tried to find some recommendations on the web but could not find anything relevant.

Let's say that I am creating a protocol buffer message that will contain a lot of fields (50+). Is it best to keep all the fields at the same level or to organize them in sub-messages? Is there any impacts on performances for one way or another?

Example:

message myMessage{
 string field1 = 1;
 string field2 = 2;
 ....
 string fieldn = n;
}

vs

message myMessage{
 SubMessage1 groupedfieldsbasedonsomebusinesslogic1 = 1;
 SubMessage2 groupedfieldsbasedonsomebusinesslogic2 = 2;

 message SubMessage1{
  string field1 = 1;
  string field2 = 2;
  ... 
  string fieldx = x;
 } 

 message SubMessage2{
  string fieldxplus1 = x+1;
  ... 
  string fieldn = n;
 }
}

I am not considering readability so much here as there are pros and cons when deserializing to have flat data or nested data. My question is really focus on the technical impacts.

score 18 · Accepted Answer · answered Apr 05 '18 at 10:56

There is no "best" - everything is contextual, and only you have most of the context.

However! Some minor thoughts on performance:

a nested approach requires more objects; usually this is fine, unless your volumes are huge
a nested approach may make it easier to understand the object model and the relationships between certain parts of the data
a flat approach requires larger field numbers; field numbers 1-15 take a single byte header; field numbers 16-2047 require 2 bytes header (and so on); in reality this extra byte for a few fields is unlikely to hurt you much, and is offset by the overhead of the alternative (nested) approach:
a nested approach requires a length-prefix per sub-object, or a start/end token ("group" in the protocol); this isn't much in terms of extra size, but:
- length-prefixe requires the serializer to know the length in advance, which means either double-processing (a "compute length" sweep), or buffering; in most cases this isn't a big issue, but it may be problematic for very large sub-graphs
- start/end tokens are something that google has been trying to kill, and is not well supported in all libraries (and IIRC it doesn't exist in "proto3" schemas); I still really like it though, in some cases :) protobuf-net (from the tags) supports the ability to encode arbitrary sub-data as groups, but it might be awkward if you need to x-plat later

Out of all of these things, the one that I would focus on if it was me is the second one.

Perhaps start with something that looks usable, and measure it for realistic data volumes; does it perform acceptably?

Is it best to have many field in protobuf message or nested messages?

1 Answers1