Converting (parsing) google protocol buffer streams from socket

Question

I am using the following code to parse a message that was SerializedwithCodedStream on to the socket:

if ( socket->read(databuffer, size) != -1)
{
    google::protobuf::io::ArrayInputStream array_input(databuffer,size);
    google::protobuf::io::CodedInputStream coded_input(&array_input);
    data_model::terminal_data* tData = new data_model::terminal_data();
    if (!tData->ParseFromCodedStream(&coded_input))
    {
        return;
    }
    else
    std::cout << tData->symbol_name() << std::endl;
}

Here is how I serialized it:

    data_model::terminal_data tData;
    tData.set_type(1);
    tData.set_client_id("C109");
    tData.set_expiry("20140915");
    tData.set_quantity(3500);
    tData.set_strat_id("056");
    tData.set_symbol_name("BANKNIFTY");
    tData.set_time("145406340");
    tData.set_trade_id(16109234);

    int total_size = tData.ByteSize() + sizeof(int);
            char *buffer = new char[total_size];
            memset(buffer, '\0', total_size);
            google::protobuf::io::ArrayOutputStream aos(buffer,total_size);
            google::protobuf::io::CodedOutputStream *coded_output = new google::protobuf::io::CodedOutputStream(&aos);
            google::protobuf::uint32 s  = tData.ByteSize();
            coded_output->WriteVarint32(s);

            tData.SerializeToCodedStream(coded_output);

            int sent_bytes = 0;
            if ( (sent_bytes = send(liveConnections.at(i), buffer, total_size, MSG_NOSIGNAL)) == -1 )
                liveConnections.erase(liveConnections.begin() + i);
            else
                std::cout << "sent "  << sent_bytes << " bytes to " << i << std::endl;

            delete coded_output;
            delete buffer;

When I try to parse, it gives the following error at runtime:

[libprotobuf ERROR google/protobuf/message_lite.cc:123] Can't parse message of type "data_model.terminal_data" because it is missing required fields: type

But as you can see (in the second code snippet) I have set the type field. What is the problem ?

Have you tried to read data directly from `coded_output` (Just to understand if there's some transmission problem)? are you sure all bytes are sent? — Massimo Costa, Aug 17 '14 at 17:50
I checked the number of bytes, that comes out correct. As to the data contained in coded_input on the other side, this I found :`C109056" BANKNIFTY(��2201409158�B 145406340` Not sure what to make of it. Some fields are surely missing. But may be they are encoded in a way I am not able to see them ? — Chani, Aug 17 '14 at 18:01
If you post your protocol buffer code, I can test it. But at least one potential issue that's jumping out is that you're using ``sizeof(int)`` to calculate ``total_size``. Shouldn't you be using ``sizeof(char)``? I could be 100% wrong, but the problem might be that you're adding garbage bytes to the message by telling the protobuff library that it's longer than it really is. — Behram Mistree, Aug 17 '14 at 18:08
@BehramMistree Line ` int total_size = tData.ByteSize() + sizeof(int); ` right ? That is calculating size of delimiter which is an int. So I guess no problems there ? — Chani, Aug 17 '14 at 18:11
You're prepending the serialized payload with length. Can you try without it? — SNce, Aug 17 '14 at 18:12
Naah men I really cannot. I need that delimiter to know the size of payload. Other wise how will i know how many bytes i have to read ? — Chani, Aug 17 '14 at 18:13

score 3 · Accepted Answer · answered Aug 18 '14 at 02:57

3

You're ignoring the count returned by read(), other than checking it for -1. You need to use it instead of size when constructing array_input.

answered Aug 18 '14 at 02:57

user207421

305,947
44
307
483

This is correct. Moreover, note that read() does not necessarily receive a single message at a time. It may receive a partial message, or multiple messages. You therefore need to frame them by sending the size as a prefix. See: http://stackoverflow.com/a/22927149/2686899 – Kenton Varda Aug 18 '14 at 19:50

Converting (parsing) google protocol buffer streams from socket

1 Answers1

Linked