4

On Linux (Kernel 3.0.0.12; C++ with GCC 4.6.1) I want to send shortly after each other a few TCP-packets via (POSIX) send-call.

With Wireshark I can see that the packets weren't sent in separate TCP-frames, but two or more were sent together in one packet.

Is there any way to tell the system to send data from one send()-buffer in own TCP-packet? The programmflow shouldn't block at the send-call.

jman
  • 11,334
  • 5
  • 39
  • 61
Bastian Born
  • 156
  • 1
  • 13
  • 5
    TCP is a streaming protocol, not a packet-oriented protocol. You can (probably) make the sending stack send exactly what you want in terms of frames, but there is _no_ guarantee that the receiver will receive these frames as-is. So I'm not sure what you are trying to do is useful. – Mat Dec 07 '11 at 20:15
  • I think this (http://stackoverflow.com/questions/6189128/sending-tcp-frames-of-fixed-length) may be related. Not sure if what you want can be achieved through user-only code. – littleadv Dec 07 '11 at 20:16
  • You might look into using UDP instead of TCP. UDP would behave as you describe. Really depends on your application which protocol makes the most sense. – TJD Dec 07 '11 at 20:20
  • As Skjaidev correctly stated, the issue is NAGLE. You can disable it with setsockopt(TCP_NODELAY): http://en.wikipedia.org/wiki/Nagle%27s_algorithm – paulsm4 Dec 07 '11 at 20:30
  • You should tell us more about the problem you are trying to solve than about the way you think you can solve it (as with many programming questions). – curiousguy Dec 08 '11 at 16:37

4 Answers4

10

Your TCP stack is implementing Nagle's algorithm trying to increase efficiency by buffering data. Its a common optimization where the intent is amortize the cost of the 40+ byte (TCP + IP) header. Instead of sending multiple small packets, the stack buffers the data and combines them into a larger packet thereby reducing the header overhead.

TCP stacks don't buffer indefinitely though. As long as there is some un-acknowledged sent data on the connection, the sender continues to buffer. The next packet is sent out when:

  • A full packet worth of data is available to be sent OR
  • All previously sent data is acknowledged by the receiver

You can usually disable this by setting the TCP_NODELAY socket option.

setsockopt(sock_fd, IPPROTO_TCP, TCP_NODELAY, (char *) &flag, sizeof(int));
jman
  • 11,334
  • 5
  • 39
  • 61
  • 1
    It seems like even with Nagle's algorithm off (i.e. TCP_NODELAY set to 1), sometimes, TCP messages are still being buffered on the sender's side. The TCP window in my pcap seems big (i.e. 14K) that not the receiver processes too slow but something else. Any idea? Thanks in advance. – Hei Dec 25 '15 at 18:19
3

If you want each packet to be sent individually, consider using UDP. You will however have to implement your own error detection and retransmission of dropped packets, etc...

Dave Rager
  • 8,002
  • 3
  • 33
  • 52
3

You can turn off Nagle, but it'd be better to stop fighting the TCP stack. It's better to add some form of framing to your protocol, so you don't need to worry about whether your transmission units have been aggregated and/or split.

In the sender, this generally means using termination characters or giving lengths.

In the receiver, it means knowing how to read up to some termination character, and/or knowing how to use part of a received transmission unit or assemble a complete message from multiple transmission units.

I like to do this with bufsock, though it's very much a Python technology. But you likely want something similar for C++: http://stromberg.dnsalias.org/~strombrg/bufsock.html

dstromberg
  • 6,954
  • 1
  • 26
  • 27
1

"packet".

And no, that's the whole point of having a "network stack" - your Ethernet frames can be transmitted and routed independently of higher levels of the TCP/IP stack. You totally don't WANT to micro-manage this!

paulsm4
  • 114,292
  • 17
  • 138
  • 190
  • 1
    Actually, the TCP term is "segment", the TCP/IP stack doesn't send individual packets, it sends segments of a stream. – ninjalj Dec 07 '11 at 21:08
  • @ninjalj: you're right, of course :) FRAMES= h/w (e.g. Ethernet frames), SEGMENTS= TCP/IP streams, PACKETS= UDP datagrams – paulsm4 Dec 07 '11 at 21:54