1

I'm receiving messages over TCP in Erlang using gen_tcp. The stream is divided into packets by using a 4 byte length header, as specified by the {packet, 4} option. This is how I call gen_tcp:listen/2:

gen_tcp:listen(Port, [binary, inet, {active, once}, {packet, 4}]).

As you can see, I use the {active, once} option so I can fetch my packets from the process mailbox without flooding it. This works fine as long as the length header is correct. If it's not, anything can happen. So I want to deal with the possibility of erroneous packets somehow.

This is a bit tricky as I'm actually dealing with a stream. Ignoring erroneous packets would be okay, but how do I get Erlang to skip these packets and recognize following packets? How is this problem usually dealt with?

Is it better to use a delimiter? I've looked at some other packet options for gen_tcp. In particular the following:

asn1 | cdr | sunrm | fcgi | tpkt | line

The only one I really understand the meaning of is line, but I don't think that would be a good option. I'm expecting to recieve packages sent and constructed in Objective C, which I'm not familiar with, containing many different kinds of data, not only strings.

Max
  • 189
  • 7

1 Answers1

4

I don't think you can skip "bad" packets when using {packet,N}

Main reason being, what's a bad packet?

any 4 bytes can indicate size, so if someone sent: Hello\n

you actually get: 0x48656c6c as the size header, indicating the frame is: 1214606444 bytes long followed by "o\n" and then you'd hang

You can set {packet_size,1024} (or whatever) to keep you from allocating 1.2gb on "hello"

Not sure if that returns an error or drops the socket.

There's really no way to recover from bad framing in TCP since messages are arbitrarily split up in transit.

if you don't use {packet,N} then you'll have to do your own segmentation control, say you get lines of text, you need to know when a message ends.

if you control the client/server protocols, you may want to use {packet,http} or a more full fledged we solution like cowboy

Then you have the client submit http requests (body can still by binary), and the framwork deals with malformed requests for you.

If you need a bi-directionaly channel, websockets might still be ok (assuming you have a websocket lib in objective c)

David Budworth
  • 11,248
  • 1
  • 36
  • 45
  • Thanks for your reply, David. I think I'll stick with length frames and rely on the protocol being correctly implemented. I was thinking about a solution where the socket's buffer would be flushed after some period of time in the `receive` clause, indicating that either no packet was received or the next packet in the buffer was erroneous, and let the client re-send the messages. I haven't figured out a way to flush a socket's buffer in active mode, though. I tried switching to `passive` mode and calling `recv/2`, hoping that it would do the trick. It didn't. (Why?) – Max Apr 22 '14 at 16:03
  • If you are using {packen,N}, you can't flush it.. If you implemented framing yourself, you can always timeout on the socket read. Flushing the buffer is never really a good option though... say your timeout is 5 seconds, and your client sends 3000 bytes. Your MTU is probably 1500 bytes, so you get two 1500 byte packets. but what if the other end hangs. Say it's on a VMWare node that gets live migrated, or you have some kind of GC pause or something. you will flush the first 1500 bytes only to get another 1500 byte bad packet. – David Budworth Apr 23 '14 at 01:43