26

How can be an array of double (1D) stored using protocol buffer? What about multi-dimensional (2D or 3D) dense arrays?

osgx
  • 90,338
  • 53
  • 357
  • 513
  • possible duplicate of [Using ProtoBuf-Net, how to (de)serialize a multi-dimensional array?](http://stackoverflow.com/questions/4090173/using-protobuf-net-how-to-deserialize-a-multi-dimensional-array) – arkon Aug 09 '15 at 01:31
  • b1nary.atr0phy, this question has solution for 1D arrays; this question also about C/C++, not about .NET. If we register my question as duplicate, will answers be merged? – osgx Aug 09 '15 at 07:34

3 Answers3

33

One could simply mimic the C/C++ memory layout:

message DoubleMatrix {
  required uint32 rows = 1;
  required uint32 cols = 2;
  repeated double data = 3 [packed=true];
}

To access the data, use data[i*cols+j] (row-major), or data[i+rows*j] (column-major). For square matrices only one of rows/cols has to be stored. Technically even in the rectangular case protobuf will know the length of the data, and the other value can be derived.

For ease of use one would probably wrap the Matrix in C++ with an Adapter class that allows access via double MatrixAdapter::get(int row, int col); it could also verify data_size()==rows()*cols().

smilingthax
  • 5,254
  • 1
  • 23
  • 19
31

An array of double would be best stored via

repeated double foo = 5 [packed=true];

repeated makes it act as a list, allowing multiple items; packed avoids a header per item.

There is no direct support for rectangular (or higher) arrays in protobuf. The closest is to store something like:

repeated innerType foo = 5; // note, can't be "packed"

message innerType {
    repeated double foo = 1 [packed=true];
}

this is broadly akin to a jagged array, but with an element between each tier.

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • 3
    Please, translate the "broadly akin to a jagged array" into more programming english? – osgx Jul 26 '11 at 05:13
  • 6
    @osgx rather than an "array of arrays", it would be more like an "array of objects, each of which *has* an array" - better? – Marc Gravell Jul 26 '11 at 05:20
  • 1
    @osgx no, that would be duplication; protobuf will take charge of lengths – Marc Gravell Jul 26 '11 at 06:52
  • 1
    while this is a correct solution, it uses more storage than storing the matrix as a 1D array as @smilingthax suggested in an alternative answer. – mimarcel Jan 21 '22 at 11:25
1

If your main goal is a denser JSON representation, you could use the "well known type" [google.protobuf.ListValue][1] for the second dimension:

import "google/protobuf/struct.proto";
message DoubleMatrix {
  uint32 cols = 1;
  uint32 rows = 2;
  repeated google.protobuf.ListValue values = 3;
}

This will produce an array of arrays when marshaling the data to JSON.

{
  "cols": 2,
  "rows": 2,
  "values": [[1, 2], [3, 4]]
}

The downside is that the ListValue type internally uses an inner type that uses the one-of pattern for the values. This may be more cumbersome to handle in your source code than using your own inner type without all the magic.

[1] https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.ListValue

ftl
  • 3,014
  • 1
  • 24
  • 25