How to do protobuf message cross-referencing?

Question

I use the below protobuf message to represent a tree structure

message Foo {
  uint32 value = 1;
  repeated Foo children = 3;
}

Now I want to add two children to a node, I basically do

Foo foo;
Foo *c1 = foo->add_children();
Foo *c2 = foo->add_children();
c1->set_value(1);
c2->set_value(1);

Because child 1 and child 2 are identical, duplicating the data consumes a lot of storage (in practice, I have huge satellite data for each node), is there anyway that I can let child 2 be a pointer (or reference) to child 1 to save the space. For example, like the below pseudo-code does

Foo foo;
Foo *c1 = foo->add_children();
foo->add_children_with_pointer(c1); // doesn't work

score 0 · Answer 1 · answered Jan 21 '20 at 07:27

There are two different ways of interpreting the question here; at the actual transport level, there is no concept of cross-referencing; the payloads will be duplicated in the output.

In terms of the in-memory representation; it is largely going to be implementation/library-specific. In the C# versions, for example: you'd just... add the same object reference again, and it should work. The C++ version, though, has a much stronger ownership model due to the arenas implementation. You could try AddAllocated, but I have a hunch that this isn't going to work either. There are a few items there under "Advanced memory management", but I'm not convinced that your scenario is a supported one.

As a side note, even in libraries where we had a model with the same object used twice: we need to keep in mind that when deserializing that data, we'll still get two different instances, not one instance with the same reference in two places - as there is no "object id" (or similar) as part of protobuf. From the deserializer's perspective, this is just two completely different objects that happen to have the same contents. Meaning: even if you get it to work when serializing; your efforts will not be respected when deserializing. So: in many ways, it isn't a useful endeavor - you will, after all, presumably be wanting to deserialize the data, and at that point: you're back where you didn't want to be.

How to do protobuf message cross-referencing?

1 Answers1