0

I have a message with repeated field, assuming the repeating field size is very large, and then I save the proto object to string (SerializeToString), which size is 200M.

message ChunkBody {
  repeated SingleMessage messages = 1;
}

Now If I want to parse this object (ParseFromString), I have to read the whole 200M file before I can parse it?

Actually I want to implement the function of iteratively reading repeated messages from file.

the environment I want to use

lib: protobuf
language : python
daohu527
  • 452
  • 4
  • 15
  • 1
    Whether you can parse in a streaming fashion is up to the protobuf parsing library in your particular language. You need to specify the language/library that you're using. That said, I haven't heard of a library that supports that. Easiest way is to split your data up into multiple protobuf messages. Like, how is suggested here: https://stackoverflow.com/questions/21974718/stream-data-into-protobuf-parser-c – Erwin Bolwidt Apr 11 '22 at 03:10
  • Thanks for the suggestion, I'm limited to "repeated" fields can read separate? Of course, it is also feasible to store a single message separately as an alternative – daohu527 Apr 11 '22 at 06:19

0 Answers0