2

I have a proxy service that translate protobuf into another struct. I can just write some manual code to do that, but that is inefficient and boilerplate. I can also transform the protobuf data to JSON, and deserlize the JSON data into the destination struct, but the speed is slow and it is CPU heavy.

The Unmarshaler interface is now deprecated, and Message interface have internal types which I cannot implement in my project.

Is there a way I can do this now?

silverwen
  • 287
  • 3
  • 11
  • In part it depends on what that other struct is for. Often the reason for having to do this is because the destination struct is actually part of another serialisation technology and is also autogenerated by that technology's tools. In that case, it is an annoying problem to solve. In languages with reflection one might be able to use that to match field to field. Otherwise the best answer is to get rid of the dest struct altogether and exclusively use the struct autogenerated by the protobuf compiler (I'm assuming that you are compiling a .proto file) – bazza Aug 03 '22 at 05:59
  • You are right. It is used by another proto and is autogenerated by my code. But I don't think i can use protobuf generated code there – silverwen Aug 03 '22 at 15:52
  • Ah, does that mean there are two separate proto files, one you control, one you don't? And both proto files define basically the same message/struct? If so, you need to get rid of your definition of the struct/message in your proto file, and use the import keyword to bring the definition in the other proto file into yours. See the import keyword in https://developers.google.com/protocol-buffers/docs/proto3. Or, is the problem of a different nature? – bazza Aug 03 '22 at 18:07
  • Or is it that you have a proto file, but there's something else that cannot use protobufs and so you're having to transcribe between the two? Most platforms and languages, even small embedded platforms, can use GPB, but there are exceptions and reasons. – bazza Aug 03 '22 at 18:13
  • BTW I saw that Go supports reflection, so one could use that. So long as the struct field names match in the two definitions, you could loop through the fields in one struct, find a field of the same name in the other struct, and copy the field value from one to another. That would be a universal transcriber, and it would adjust if the struct definitions changed. – bazza Aug 03 '22 at 18:17
  • One use proto and grpc, but the other is not and using other generated go structure. I have an idea about how to translate between the two, but I don’t know which API I can implement to do this after the removal of custom unmarshaler interface. – silverwen Aug 04 '22 at 19:17
  • Ah, the hardest problem to deal with. I would have a go at using Go's reflection as I've suggest above. I don't know Go at all, so I've no idea if that can actually work, but I've done something similar in C#, where it worked a charm. – bazza Aug 05 '22 at 06:26

1 Answers1

0

Psuedo code: basically, if Go's reflection supports setting and getting of struct / class fields by some sort of field identifier, then you can do this. Something like this in C# works, so long as the field types in the two classes are the same (because in C#, I'm doing object = object, which ends up being OK if they're the same actual type).

SourceStructType sourceStruct;
DestStructType destStruct;

foreach (Field sourceField in sourceStruct.GetType().GetFields())
{
    Field destField = destStruct.GetType().FindFieldByName(sourceField.name);
    destStruct.SetFieldValue(destField) = sourceStruct.GetFieldValue(sourceField);
}

If the structs are more complex - i.e. they have structs within them, then you'll have to recurse down into them. It can get fiddly, but once written you'll never have to write it ever again!

bazza
  • 7,580
  • 15
  • 22