Why iperf bandwidth is highter than dd->netcat

Question

I want to measure actual speed of 1Gb Ethernet interface on my embedded device with ARM cortex-A8 two-core 600MHz processor, connected to PC on the other side.

I've tried to check it with iperf client, installed on device and iperf server, installed on PC. And I got the following results

Than I changed iperf client on following command:

$ dd if=/media/sd0p1/Videos/20180221_164356.mp4 bs=1M | nc 192.168.0.2 5001

And in this case bandwidth is approximately 2 times less than with iperf: dd-nc-results.

I've also read this post, and than executed 2 instances of dd->nc pipeline on the client side, that sends data to iperf on the PC side and got this result. Also I figured out, that in case of dd->nc my CPU loads nearly full. I've got two questions:

1. Which of that cases I can consider as more accurate way to identify true bandwidth.
2. How this results will vary, when I start to launch more apps on my device? Am I right, that in case of increasing number of working processes, bandwidth should to go down.

Answer 1

I experienced the same phenomenon, in a similar setup (ARM <-> PC). I Googled, and found your question - with no answer. So I proceeded to do some measurements, and saw that in the tiny-brained ARM I tested this on, iperf3 consumed 22% of the CPU during the test - but in the dd ... | nc version, netcat consumed 85%:

So there is clearly a lot more overhead in the scenario involving big amounts of data passing over the pipe. My guess is because of the small CPU cache on the ARM CPU, the crossing over of the dd-generated data from one process to another is enough to spillover the CPU cache size - with a big impact on final data transfer speeds (in my case, 50%).

So, the answer to your 1st question, is that the two test paths give you different information:

• The iperf3 test checks your network speed. Only that.
• The dd ... | nc test also checks how your network speed behaves under some load on your ARM CPU. In both your setup and mine, it clearly suffers.

As to your 2nd question - well, if you launch even more apps that use the ARM CPU, then your top speed will suffer even more.