How are we supposed to use linux bonding mode balance-rr?

Question

I'm trying to setup 10 servers with a quad port gigabit nic using the linux bonding driver in balance-rr mode. But how is this supposed to be done?

I'm not buying 10Gbe hardware now because it's too expensive at this moment. So, I wanted to use multiple gigabit nics and aggregate the bandwidth.

According to the linux bonding driver's documentation the balance-rr mode is the only mode that will stripe traffic across multiple interfaces and should, in theory, get me about 4 gigabits?

So, how is this supposed to be configured on the switches?

I have a HP 2810-48G switch and it supports trunk groups. According to the bonding driver's documentation that's what I need for the balance-rr mode to work, right?

But if I configure trunk groups like that it might work (I haven't tried that (yet)) and the server will probably be able to go above 1 gigabit in throughput.

But the switch won't be able to strip that traffic across the interfaces of the second server, does it? At least, there is nothing in the manual of the switch about round-robin striping of traffic. In fact, I can't even find that in manuals of newer switches either...

So, how am I supposed to configure this stuff so that I can get close to 4gigabit/sec in a single TCP session?

Maybe I need 4 vlans and put each port of each server in a seperate vlan? That sounds like a possibility. Again, I haven't tried that yet, because it is not even a suggestion in the bonding driver's documentation.

Has anyone got any idea how I'm supposed to do this right?

Thanks!

Answer 1

According to documentation, the HP 2810-48G supports both LACP (802.3ad) and "non-protocol" trunking/bonding (the naming differences get even more confusing when mixing network gear vendors!).

HP describes the "non-protocol trunking":

This method creates a trunk group that operates independently of specific trunking protocols and does not use a protocol exchange with the device on the other end of the trunk. With this choice, the switch simply uses the SA/DA method of distributing outbound traffic across the trunked ports without regard for how that traffic is handled by the device at the other end of the trunked links. Similarly, the switch handles incoming traffic from the trunked links as if it were from a trunked source.

(SA/DA is source address/destination address. I'm guessing this compares to the mode "balance-xor" in Linux bonding. The HP documentation doesn't specify whether it's using MAC- or IP-address, but I'm assuming it's using MAC-addresses. How much this actually balances your load is dependent on your network topology.)

I believe that your setup will work if you configure balance-rr on the Linux end and non-protocol trunking on the switch end. You will however see different load balancing on each end, with traffic from the switch not utilizing all ports.

To utilize all ports you might have to test the setup, and look into application/protocol specific solutions like multipathing or distributing clients over more than one server IP-address for example.