2

I have two Linux machines that I wish to connect via a bonded link.

One machine has two UMTS modems (DN:5mbit UL:1.2mbit) as its gateways, and the other machine has optical fiber (DL:100Mbit UL: 20mbit) as its gateway.

I can successfully create 2 OpenVPN channels (one per UMTS modem, using iptables rules) and have applied the Linux bonding driver (mode 0, round robin distribution) to these channels.

So far, so good. The bonded interface connects the two Linux machines via two aggregated VPN channels, where I can ping each machine, transfer files, etc, etc.

My problem is the bandwidth of the bond. Theoretically the bandwidth should double, but in fact, it is the same, independent of the number of VPN connections inside the bond.

In the machine with two UMTS modems, when I use only one VPN, I can reach the other machine with bandwidth near DN:5mbit UL:1.2mbit. Using 2 VPNs inside the bond interface the bandwidth per channel is near DN:2.5mbit UL:0.6mbit, so the overall bandwidth is the same, whether I use one or both VPN channels.

This behavior happens when I transfer data using either TCP or UDP, so it isn't a protocol issue.

Has anyone else experienced this too?

Thanks in advance.

asciiphil
  • 3,086
  • 3
  • 28
  • 53
frico
  • 49
  • 1
  • 6
  • How are you testing bandwidth? Bonding does not in-fact double bandwidth. Bonding will not give any additional for a single connection between two systems. Bonding distributes connections between the links in the bond. So a connection from A to B will be on link one, and a connection from A to C will be on link two. – Zoredache Jun 03 '13 at 16:16
  • 1
    Your observation is right for all bonding modes, except bonding mode 0, that is what i am using. In bonding mode 0 the packets are distributed in a round robin manner for all slaves inside the bonding, so theoretically if the links are perfectly symmetrical, bandwidth aggregation should happen. I am testing bandwidth using iperf tool and visualizing the traffic with iptraf tool. – frico Jun 03 '13 at 16:32
  • 1
    Right, didn't notice you were in mode 0. Still curious how you are testing though. – Zoredache Jun 03 '13 at 16:34
  • 1
    I am testing bandwidth using iperf tool and visualizing the traffic with iptraf tool. In iptraf i can visualize the data rates on each VPN and other physical interfaces – frico Jun 03 '13 at 16:35
  • Testing the bandwidth with a file transfer lead to the same results. – frico Jun 04 '13 at 13:41

3 Answers3

2

Finally i found the (obvious) cause of the problem.

"If you have a network link with low bandwidth then it's an easy matter of putting several in parallel to make a combined link with higher bandwidth, but if you have a network link with bad latency then no amount of money can turn any number of them into a link with good latency." @ It's the Latency, Stupid, Stuart Cheshire, May 1996.

Since the VPN latency is around 110ms, the max bandwidth of the link would be around 4.8mbps (acording to http://bradhedlund.com/2008/12/19/how-to-calculate-tcp-throughput-for-long-distance-links/)

To check this i put in practice a test, i lowered each UMTS modems bandwidth to 2mbps DL and 1.2mbps UL (shaping traffic with wondershaper), resulting 3.65mbps DL and 2mbps UL of aggregated bandwidth.

Resuming, i can use VPN bonding if the aggregated bandwidth don't exceed the maximum bandwidth imposed by the latency, otherwise it will be waste of resources.

frico
  • 49
  • 1
  • 6
  • There's no such thing as the "maximum bandwidth imposed by the latency": A explained by your link, the maximum bandwidth is contrained by latency times TCP window size (amount of in-flight Bytes). Since you control both sides of the connection, you can simply increase the TCP window sizes. Example values for 10 Gbit/s over a 200 ms latency connection: Set the `sysctl`s `net.core.wmem_max` and `net.core.rmem_max` to `1073741824` each, and set `net.ipv4.tcp_rmem` and `net.ipv4.tcp_wmem` to `4096 87380 1073741824` each, on both sides. Additionally, enabling BBR congestion control also helps. – nh2 Mar 22 '21 at 22:44
1

I doubt return traffic with data is coming balanced between links, so would recommend to test an uplink speed on bonded interface first.

Mik Ko
  • 11
  • 1
0

Why don't you just increase the TCP window size, like your link says? If you double the size, you should be able to reach the desired bandwidth.

asciiphil
  • 3,086
  • 3
  • 28
  • 53
cclecle
  • 1
  • 1