1

I'm currently working in high performance computing and the protocol we choose to connect the backend applications is flatbuffers because binary and zero-copy...

In one end, an app is going to send large amounts of data in "client streaming" form, but as we know, grpc has a default package limit of 4MB (can be set to a bigger size, but that is not what we want). To get over that, we are creating chunks of the data to send, these chunks size must be less than 4MB each but currently we don't know how to measure the size of the chunk BEFORE the flexbuffer is "finished".

We want to know the size of the flexbuffer chunk before is finished because we don't actually know how much data is going to be transferred prior to the construction of the chunk (we are using flexbuffers here). Then, if we know the size, we can stop the building and sent the chunk before it reaches grpc's max package size.

Update

I'm sorry I confused the terms. What we want is to know the flexbuffer size before is finished.

Thank you in advance! Hope everyone is doing good.

GChiappe
  • 71
  • 5
  • In Java, `FlatBuffersBuilder.offset()` gives the current size at any time, similar to `FlatBuffersBuilder::GetSize()` in C++. – Aardappel Jun 07 '22 at 04:21
  • Thank you, but I confused terms yesterday, what we want is to know the flexbuffer size before is finished (while building it) – GChiappe Jun 07 '22 at 14:17
  • Hi @Aardappel I am not able to find offset method on FlatBuffersBuilder. I am using 23.3.3 version of flatbuffers-java. Is it available in specific version? – G.G. Mar 10 '23 at 09:24

0 Answers0