4

I transfer data between two Ubuntu Servers (12.04) via iperf on 1Gb LAN.

iperf -s    <->     iperf -c <addr> -n2G
TCP speed = 925 Mbits/sec

iperf -su   <->     iperf -uc<addr> -b1G -n2G
UDP speed = 810 Mbits/sec
Jitter = 0.016 ms
Lost = 0 .. 0.01%

I expected that UDP is much faster than TCP. Could anyone please explain why does UDP slower than TCP on Ubuntu Server?

pehrs
  • 8,789
  • 1
  • 30
  • 46
Rom098
  • 198
  • 1
  • 3
  • 8
  • It could be anything. Your testing methodology and setup is probably just the cause. – Ben Lessani Sep 26 '12 at 10:49
  • sonassi, do you know a methodology that is better/more accurate than mine? – Rom098 Sep 26 '12 at 11:03
  • Not really - you haven't mentioned what you actually plan on doing. ie. why you need UDP to be faster. Unless of course, your intention was just to run `iperf` - in which case, its the perfect test. Your testing should reflect the real world load - otherwise the results can be somewhat meaningless. – Ben Lessani Sep 27 '12 at 00:08
  • 1
    I'm trying to determine maximun UDP transmission speed on my Linux servers, Ubuntu 12.04 on my 1GB LAN. Could you please suggest any appropriate approach? – Rom098 Sep 27 '12 at 10:26
  • 1
    No I can't. **You haven't mentioned what you actually plan on doing**. I think your missing the point of what I'm saying. Make the test reflective of what your are actually trying to do. So if your future plans for the servers is just to run `iperf` tests continuously, then your methodology is fine. If your future plans is to run OpenVPN over UDP - then your testing methodology is useless. All you're getting at the moment is a throughput figure for `iperf` - but that ***is not*** the same as what your actual application throughput may be. – Ben Lessani Sep 27 '12 at 10:38

3 Answers3

12

Each frame goes through several buffers as you send it: The application buffer, The Protocol Buffer, The Software interface buffer and the Hardware interface buffer. As you start stressing the stack by sending high speed data you will fill up these buffers and either block or lose data. You also have strategies for timeliness and polling that can impact your performance. For example, by using a larger buffer and poll less often you can get much better performance while sacrificing latency.

TCP is optimized for high speed bulk transfers while UDP is optimized for low latency in the Linux kernel. This has an impact on buffer sizes and how data is polled and handed over. In addition to this, you frequently have offloading to hardware for TCP. I would expect considerably better performance for TCP compared to UDP.

Note that sending high speed data over UDP is usually a bad idea, unless you implement your own congestion control. TCP protects your network from congestion collapses. Use UDP when you have small amounts of data or high timeliness requirements.

pehrs
  • 8,789
  • 1
  • 30
  • 46
10

As it is said above, iperf limits itself by design. In src/Client.cpp, method 

void Client::Run( void )

calls

ReportPacket( mSettings->reporthdr, reportstruct );

after writing each UDP datagram.

ReportPacket() is rather slow and it slows the whole thing.

In iperf3, burst writes are introduced for UDP, you can specify number of burst writes in -b command line parameter, like -b 10240M/2000

otherwise, you can use netperf with -t UDP_STREAM.

Ladadadada
  • 26,337
  • 7
  • 59
  • 90
Sergey
  • 101
  • 1
  • 2
  • I tried both params but still see `iperf` UDP tests run about 10000X slower than TCP (1Mbit/s vs 10Gbit/s). – sudo Jan 26 '17 at 23:05
8

The problem is inside iperf itself. When you send a UDP flow iperf will do some internal bookkeeping for each datagram in the flow. On the other hand when you send a TCP stream iperf does bookkeeping only once per stream. In my measurements it came out that iperf-UDP wastes a lot of time doing the per-datagram measurements hence the slower performance.

If you really want to compare TCP vs UDP performance is better to write your own test program. and make sure that the amount of computations is the same in both cases.

berebere
  • 81
  • 1
  • 1