what's the right way to do polymorphism with protocol buffers?

Question

I'm trying to long-term serialize a bunch of objects related by a strong class hierarchy in java, and I'd like to use protocol buffers to do it due to their simplicity, performance, and ease of upgrade. However, they don't provide much support for polymorphism. Right now, the way I'm handling it is by having a "one message to rule them all" solution that has a required string uri field that allows me to instantiate the correct type via reflection, then a bunch of optional fields for all the other possible classes I could serialize, only one of which will be used (based on the value of the uri field). Is there a better way to handle polymorphism, or is this as good as I'm going to get?

Answer 1

In proto3 the extend keyword has been replaced. From the docs: If you are already familiar with proto2 syntax, the Any type replaces extensions.

syntax = "proto3";

import "google/protobuf/any.proto";

message Foo {
  google.protobuf.Any bar = 1;
}

But beware: Any is essentially a bytes blob. Most of the times it is better to use Oneof:

syntax = "proto3";

message A {
    string a = 1;
}

message B {
    string b = 1;
}

message Foo {
  oneof bar {
    A a = 1;
    B b = 2;
  }
}

Answer 2

There are a few techniques for implementing polymorphism. I try to cover them all here: Protocol Buffer Polymorphism

My preferred approach uses nested extensions:

message Animal
{
    extensions 100 to max;

    enum Type
    {
        Cat = 1;
        Dog = 2;
    }

    required Type type = 1;
}

message Cat
{
    extend Animal
    {
        required Cat animal = 100; // Unique Animal extension number
    }

    // These fields can use the full number range.
    optional bool declawed = 1;
}

message Dog
{
    extend Animal
    {
        required Dog animal = 101; // Unique Animal extension number
    }

    // These fields can use the full number range.
    optional uint32 bones_buried = 1;
}

Answer 3

Jon's solution is correct and working but pretty weird (for me). But Protocol Buffers is quite simple, so You can do something like that:

enum Type {
    FOO = 0;
    BAR = 1;
  }

message Foo {
  required Type type = 1;
}

message Bar {
  required Type type = 1;
  required string text = 2;
}

Basically message Bar extends message Foo (from practical side of course). Implementation in Java is simple too:

Bar bar = Bar.newBuilder().setType(Type.BAR).setText("example").build();
byte[] data = bar.toByteArray();

----

Foo foo = Foo.parseFrom(data);
if(foo.getType() == Type.BAR){
   Bar bar = Bar.parseFrom(data);
   System.out.println(bar.getText());
}

I known, it's not an elegant solution, but it's simple and logical.

Answer 4

Check out Extensions and Nested Extensions for a slightly cleaner way to do this.

Answer 5

Have you considered using extensions? You could have your uri field determine the type to use and then just load the appropriate extensions. If you know your fields are mutually exclusive then you could reuse the field id between separate extensions.

You have to handle this all yourself because protocol buffers aren't designed to be self describing beyond a simple list of values. This is touched on in the google techniques page.

Answer 6

A solution a little better, for me, that the @Łukasz Marciniak's answer.

If Bar extends Foo, simply write:

message Bar {
   optional Foo foo = 1;
   optional double aDouble = 2;
}
message Foo {
   optional string aString = 1;
}

So if Foo evolves only Foo message is modified.