3

I am looking into using Google Protobuffers for delta messaging. Meaning I only want to send out the changed values of my domain object.

But that exposes a problem with the protocol for this purpose. I can easily just omit the properties that have not changed, and that will present us with a compact message.

But what about properties that change value from _something_ to null? There is no way to distinguish between these two scenarios in a protocol buffer.

What have others done here? I am looking at a few different solutions:

  1. Add a meta property to all objects, that is an array of int. In case any of the properties should change to null, include the field number in this array. If no properties change, then the meta property is just omitted and doesn't take up bandwidth in the message.

  2. Add a meta property that is a bit mask, but works like the array mentioned in option 1. This might be harder for clients to understand though.

  3. Use a standard way that I haven't been able to find yet.

BR Jay

Jay Pete
  • 4,123
  • 4
  • 35
  • 51

2 Answers2

2

Protobuf 3 isn't very well suited for this. But in protobuf 2, you can have a field that is present but has value of null.

Because protobuf 2 isn't going to disappear any time soon, I'd suggest just use that for this kind of purposes.

jpa
  • 10,351
  • 1
  • 28
  • 45
2

I just wanted to post a follow-up on this and explain what I did.

As @jpa correctly pointed out protobuffers are not made for delta-compression.

So the way I solved it was to use some meta properties and rely on that convention. I have a close partnership with the people consuming the data, so conventions can be agreed upon.

  1. Values that are set specifically to null

    I have added an int array to the messages. This int array is empty most of the time and does not have an impact on the message size. When a property is set to null, I will add the property tag to this array and that way indicate that it has specifically been set to null in that message update.

  2. Arrays that are emptied

    This works in the same way as the nulls array. I have added an int array to the messages. This int array is empty most of the time and does not have an impact on the message size. When an array is emptied, I will add the property tag to this array and that way indicate that it has specifically been emptied that message update.

  3. Objects that are deleted

    To indicate that an object has been deleted, I have added a boolean property indicating that the object has been deleted. When the object is deleted I will set this value to true, and otherwise null, so it doesn't take up space in the message. The resulting message is the key identifier for that object and the boolean indicating that it is deleted.

It requires that the convention is understood by the clients but otherwise it works pretty well.

Jay Pete
  • 4,123
  • 4
  • 35
  • 51