nanopb (Protocol Buffers library) repeated sub-messages encode

Question

we are using the nanopb library as our Protocol Buffers library. We defined the following messages:

simple.proto:

syntax = "proto2";

message repField {
    required float x = 1;
    required float y = 2;
    required float z = 3;
}

message SimpleMessage {
    required float lucky_number = 1;
    repeated repField vector = 2;
}

with simple.options

SimpleMessage.vector        max_count:300

So we know the repField has a fixed size of 300 and thus defining it as such.

Parts of the generated one looks like:

simple.pb.c:


const pb_field_t repField_fields[4] = {
    PB_FIELD(  1, FLOAT   , REQUIRED, STATIC  , FIRST, repField, x, x, 0),
    PB_FIELD(  2, FLOAT   , REQUIRED, STATIC  , OTHER, repField, y, x, 0),
    PB_FIELD(  3, FLOAT   , REQUIRED, STATIC  , OTHER, repField, z, y, 0),
    PB_LAST_FIELD
};

const pb_field_t SimpleMessage_fields[3] = {
    PB_FIELD(  1, FLOAT   , REQUIRED, STATIC  , FIRST, SimpleMessage, lucky_number, lucky_number, 0),
    PB_FIELD(  2, MESSAGE , REPEATED, STATIC  , OTHER, SimpleMessage, vector, lucky_number, &repField_fields),
    PB_LAST_FIELD
};

and part of simple.pb.h:

/* Struct definitions */
typedef struct _repField {
    float x;
    float y;
    float z;
/* @@protoc_insertion_point(struct:repField) */
} repField;

typedef struct _SimpleMessage {
    float lucky_number;
    pb_size_t vector_count;
    repField vector[300];
/* @@protoc_insertion_point(struct:SimpleMessage) */
} SimpleMessage;

We try to encode the message by doing:

// Init message
SimpleMessage message = SimpleMessage_init_zero;    
pb_ostream_t stream = pb_ostream_from_buffer(buffer, sizeof(buffer));

// Fill in message
[...]

// Encode message
status = pb_encode(&stream, SimpleMessage_fields, &message);

// stream.bytes_written is wrong!

But the stream.bytes_written is wrong which means it is not encoded correctly, although status=1.

In the documentation for pb_encode() it says:

[...] However, submessages must be serialized twice: first to calculate their size and then to actually write them to output. This causes some constraints for callback fields, which must return the same data on every call.

But, we are not sure how to interpret this sentence - what steps to follow exactly to achieve this.

So our question is:

What is the correct way to encode messages that contain fixed-size (repeated) submessages using the nanopb library?

Thank you!

score 2 · Accepted Answer · answered Jun 25 '19 at 07:01

You're not using callback fields here, so that quote doesn't matter for you. But if you were, it would just mean that in some situations your callback would be called multiple times.

Are you the same person as on the forum? Your stack overflow question does not show it, but the person on the forum has a similar problem that appears to be due to not setting vector_count. Then it will remain as 0 length array. So try adding:

message.vector_count = 300;

In the future, please wait a few days before posting the same question in multiple places. It's a waste of volunteer time to answer the same question multiple times.

nanopb (Protocol Buffers library) repeated sub-messages encode

1 Answers1