2

I'm considering using gob ("encoding/gob") for serializing data in a network protocol, I have been searching around and can't seem to find any solution to these problems:

Message framing - The gob documentation gives the impression that you can simply wrap your TCP connection in a gob decoder and read away. But what happens if you only received half a message? Can gob somehow deal with this or am I forced to add a message frame and copy over the message data into a buffer for gob to unserialize?

Different types of messages - The protocol has different types of messages, how is this best handled with gob? By having an identifier before every gob blob indicating the type of the data? By putting all messages into a "Master" message which contains fields for all the different messages (reducing it to only one type of message)? I tried the latter (simpler) and it seems to have a HUGE overhead (>650 bytes).

Cshark
  • 101
  • 6
  • I tried encoding a struct using gob, took the output, copied it into a buffer twice and removed the last ~5 bytes to simulate a half transfered structure. This gives me an error, yet empties the buffer. Am I wrong in my assertion that this could happen? I did not expect gob to be adding framing. – Cshark Jun 29 '15 at 14:51
  • I think I'm an idiot! The overhead that I described only occurs once (describing the structures) and framing is added? I attempted to simulate the last structure being half send. I think I completely misunderstood gob - expected it to work like a protocol buffer. – Cshark Jun 29 '15 at 15:02
  • Also does the .Decode method block when the next value is not complete yet? E.g Only the first part of a large structure has been transfered. – Cshark Jun 29 '15 at 15:07
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/81881/discussion-between-cshark-and-thundercat). – Cshark Jun 29 '15 at 15:17

1 Answers1

3

The gob documentation gives the impression that you can simply wrap your TCP connection in a gob decoder and read away.

Correct. The package is designed to stream multiple values between an encoder and a decoder.

But what happens if you only received half a message?

The decoder calls the underlying io.Reader to get data. If reader cannot return data, then the reader will return an error. The decoder returns this error to the application.

There's no way to recover the decoding stream if the io.Reader returns an error.

Different types of messages

You can encode pairs of messages where the first tells the application the type to expect in the second.

You can also create a "master" type as you describe. The overhead that you see is incurred once per stream, not once per value.

Charlie Tumahai
  • 113,709
  • 12
  • 249
  • 242