14

I'd like to know the general cost of creating a new connection, compared to UDP. I know TCP requires an initial exchange of packets (the 3 way handshake). What would be other costs? For instance is there some sort of magic in the kernel needed for setting up buffers etc?

The reason I'm asking is I can keep an existing connection open and reuse it as needed. However if there is little overhead reconnecting it would reduce complexity.

seand
  • 5,168
  • 1
  • 24
  • 37
  • 2
    I think the latency of the handshake is the most significant cost. – CodesInChaos Jan 29 '11 at 23:28
  • Ahh good point. The connection isn't considered open until the entire handshake completes. However once open you can stream out data without waiting for the ack on every segment (because of the sliding window) – seand Jan 29 '11 at 23:32
  • there is no 'ack on every segment'. – user207421 Jan 29 '11 at 23:40

3 Answers3

11

Once a UDP packet's been dumped onto the wire, the UDP protocol stack is free to completely forget about it. With TCP, there's at bare minimum the connection details (source/dest port and source/dest IP), the sequence number, the window size for the connection etc... It's not a huge amount of data, but adds up quickly on a busy server with many connections.

And then there's the 3-way handshake as well. Some braindead (and/or malicious systems) can abuse the process (look up 'syn flood'), or just drop the connection on their end, leaving your system waiting for a response or close notice that'll never come. The plus side is that with TCP the system will do its best to make sure the packet gets where it has to. With UDP, there's no guarantees at all.

Community
  • 1
  • 1
Marc B
  • 356,200
  • 43
  • 426
  • 500
8

Compared to the latency of the packet exchange, all other costs such as kernel setup times are insignificant.

user207421
  • 305,947
  • 44
  • 307
  • 483
4

OPTION 1: The general cost of creating a TCP connection are:

  1. Create socket connection
  2. Send data
  3. Tear down socket connection

Step 1: Requires an exchange of packets, so it's delayed by to & from network latency plus the destination server's service time. No significant CPU usage on either box is involved.

Step 2: Depends on the size of the message.

Step 3: IIRC, just sends a 'closing now' packet, w/ no wait for destination ack, so no latency involved.

OPTION 2: Costs of UDP:*

  1. Create UDP object
  2. Send data
  3. Close UDP object

Step 1: Requires minimal setup, no latency worries, very fast.

Step 2: BE CAREFUL OF SIZE, there is no retransmit in UDP since it doesn't care if the packet was received by anyone or not. I've heard that the larger the message, the greater probability of data being received corrupted, and that a rule of thumb is that you'll lose a certain percentage of messages over 20 MB.

Step 3: Minimal work, minimal time.

OPTION 3: Use ZeroMQ Instead

You're comparing TCP to UDP with a goal of reducing reconnection time. THERE IS A NICE COMPROMISE: ZeroMQ sockets.

ZMQ allows you to set up a publishing socket where you don't care if anyone is listening (like UDP), and have multiple listeners on that socket. This is NOT a UDP socket - it's an alternative to both of these protocols.

See: ZeroMQ.org for details.

It's very high speed and fault tolerant, and is in increasing use in the financial industry for those reasons.

Kevin J. Rice
  • 3,337
  • 2
  • 24
  • 23
  • 3
    I'm not sure where you got the 20MB number for UDP. In IPV4, the max message size is 65,507 bytes (65,535 − 8 byte UDP header − 20 byte IP header), according to Wikipedia. Maybe you are assuming IPV6? – Joe Mar 18 '15 at 22:09
  • I was not referring to packet size, but to message size. Plus, the number is a hearsay heuristic. The situation: on a reasonably high-speed internal datacenter network via UDP, in any set of N messages of size 20 MB, 5% of N (N*0.05) of them were corrupted. This was a firehose w/ HUGE data one-> several, with different sets of boxes listening for messages addressed to them. Sorry for the confusion. His (my coworker's) message to me was that if he used larger chunks, a higher percentage would have errors and nothing would send. He had a backchannel for requesting re-sends. – Kevin J. Rice Mar 20 '15 at 00:04
  • 3
    UDP has no concept of a message larger than a single 64k datagram, because the size field in the UDP header is 16 bits (https://tools.ietf.org/html/rfc768). That is typically larger than the MTU of the underlying transport layer, so IP will likely fragment and reassemble a 64k packet. If any fragment is lost, the whole packet is lost, so exceeding the MTU is not recommended. A 20MB message requires a higher level protocol over UDP or something else like TCP. If you have a reference for larger messages, I am interested to read it. – Joe Mar 21 '15 at 01:34
  • 1
    ZeroMQ is not 'an alternative to both these protocols'. It 'carries messages across inproc, IPC, TCP, TIPC, multicast'. Multicast is UDP of course. You seem to think it is another IP transport protocol. It isn't. – user207421 Jan 25 '18 at 22:45