2

I have a backup server installed in rack A. That rack has a Cisco WS-C3560G-48TS-S switch installed.

I also have 10 proxmox servers (=linux vm host) in rack B. That rack has a HP 2810-48 switch installed.

There is currently only 1 CAT6 cable between the switches. It's a trunk for all vlans.

These 10 proxmox servers are doing their backups all at the same time (default config) and offcourse the link between two switches is saturated during backups.

So I need to fix that. But I'm not sure what's the best way?

My idea was to add a quad-port nic to the backupserver, setup LACP between the backup-server and the cisco switches (over 6 ports). Then also do LACP between the cisco switch and the HP switch using 6 ports. And a 7th cable for the other vlans.

But when I said that on a networking IRC channel some guy said that this won't work as expected because of switches and their buffers. I couldn't completely understand what he was trying to say and then he disappeared..

I don't see a problem in my idea, but I'm not a network professional so I'm confused now...

The guy said that the only good way to fix this is by connecting all servers to the same switch or use switches with a 10G port and connect them together using a 10G port.

Can anyone please explain to me if my idea is bad and why?

Jorne
  • 123
  • 1
  • 6
  • "I'm not a network professional" Maybe it's time to hire one? – ceejayoz Jul 09 '15 at 15:54
  • 1
    LACP is the most misunderstood networking concept out there. If you need more bandwidth, plan for 10GbE if possible. – ewwhite Jul 09 '15 at 16:07
  • LACP will work in this case, won't it? It should spread traffic over multiple links because I have multiple clients (=servers) ? – Jorne Jul 09 '15 at 16:10

1 Answers1

1

Your idea is not bad.

I have actually similar setups as yours for my backup servers. But not that much for the extra bandwidth rather the extra redundancy.

I don't know what that guy on IRC was trying to say but I know that LACP won't work as expected in terms of maximum bandwidth you can reach per server.

LACP does not bond the ports together to give a combined bandwidth for single connections for example. That's the main misunderstanding about LACP and Bonding (or Etherchannel in terms of Cisco)

So if you have 6 gbit ports bonded into an etherchannel between your 2 switches and you want to do data transfer from one server on one switch to another on the order switch it will utilize only one of those 6 gbit ports (assuming for the sake of argument that the servers on both sides have more than 1gbit available).

Now if another server from one switch want's at the same time to do some data transfer on another server on the other switch depending on your LACP configuration it will use another 1gbit port for that transfer.

So in total per server/port/whatever you will still get a maximum of 1gbit, but when you are doing parallel data transfers LACP will utilize/load balance the 6 available gbit ports resulting eventually in a total/combined bandwidth of 6 gbits.

So, in theory what you propose should work OK when running multiple backups at the same time.

But, from my personal experience all my backup servers' bottleneck is the storage rather than the network (I use 2 gbit ports instead of 4 you are suggesting).
Mechanical disks can be way slower than 1gbit when randomly accessed. Of course it has to do with your backup system and policy so that may not be an issue for you.

Cha0s
  • 2,462
  • 2
  • 16
  • 26
  • Thanks, that's what I was thinking too. I understand the LACP concept.. but that guy was telling me that it won't work because of buffering between switches.. He said I will experience a lot of "tx drops" .. but if the switches balance the load (from multiple servers) over the lacp bond a little.. it shouldn't be much of a problem I hope... – Jorne Jul 09 '15 at 16:20
  • Indeed! What you describe sounds right! Maybe he was talking about the CAM tables because of the 'extra' ports since you would be 'extending' the switch with the trunk port? But in this case I don't think that bonding would have anything to do since you already have a trunk port between switches. I do use over 15 etherchannels in a single stack (2 x 3750E) without any drops at all (and it's mainly for SAN traffic). So no problems there. – Cha0s Jul 09 '15 at 16:28
  • 1
    If you used src-mac on the etherchannel and added the extra card to the backup server but with 4 seperate IPs you should be able to utilise more then 1gbit as the src would be one of 6 servers and the destination would be 1 of 4. – Drifter104 Jul 09 '15 at 16:28