3

We are using Windows 2008 Failover Cluster for our iSCSI SAN system. There are two nodes which have 3 nics on each. Normally, these nics are working as heartbeat, LAN and iSCSI connection to SAN. The nodes have these setup:

NIC 1 => Heartbeat
NIC 2 => LAN
NIC_3 => iSCSI

The motherboards on the nodes don't have any extra PCI slot for extra nic. And we want to increase bandwith of this SAN system. The motherboards with more(4 or more) pci slot are expensive and hard to find. I realized that our nics on the nodes have VLAN option. And I've thought we can use VLAN NLB(or LAGG) for LAN connections and VLAN MPIO for iSCSI connections. Also we can spare a VLAN for heartbeat connection. Because heartbeat connection doesn't use much bandwidth, we can increase bandwidth a little bit. Of course if we can? I want to ask that it is possible. If it is possible, do you suggest that? If it is possible I want to setup VLANs like that:

NIC 1 => VLAN_1_NLB(Heartbeat), VLAN_2_NLB(LAN)
NIC 2 => VLAN_2_NLB(LAN), VLAN_3_MPIO(iSCSI)
NIC_3 => VLAN_3_MPIO(iSCSI)

or maybe:

NIC 1 => VLAN_1_NLB(Heartbeat), VLAN_2_NLB(LAN), VLAN_3_MPIO(iSCSI)
NIC 2 => VLAN_2_NLB(LAN), VLAN_3_MPIO(iSCSI)
NIC_3 => VLAN_2_NLB(LAN), VLAN_3_MPIO(iSCSI)

Is this possible or what else can you suggest? I'm not sure VLAN configuration though.

The nics have 802.1Q and the switch in the network is a managed switch.

poige
  • 9,448
  • 2
  • 25
  • 52
oruchreis
  • 179
  • 3
  • 12

3 Answers3

2

You don't specify if the system has 3 nics built-in to the motherboard or if they are add-in cards that are using up all available slots. If it is the latter case, I highly suggest you get replacement nic cards with more than one port instead of doing this vlan hackery. 2-port and 4-port server nic cards are readily available from Intel and 2-port gigabit nic is not too expensive.

Although the VLAN option will work it is important to keep the heartbeat separate. Although it is a low bandwidth connection it is latency sensitive. Combining the heartbeat and iSCSI on the same physical media would be a particularly bad idea since iSCSI tends to have bursty traffic that can saturate the link for short periods. The iSCSI traffic is also latency sensitive. However, if latency increases, you will only suffer decreased I/O performance instead of the potential of unnecessary and unintended cluster fail-over events.

If you are intent on going down the vlan path also see if your nics support QOS. In this way you can assign priority queues to traffic on different vlans. With QOS, the best scenario I can think of with 3 nics would be:

  • NIC1 -> LAN(VLAN1 - low priority QOS), Hearbeat(VLAN2 - high priority QOS)
  • NIC2 -> iSCSI
  • NIC3 -> iSCSI

In this way you would double your SAN bandwidth and should assure no disaterish cluster behavior. If you cannot use QOS I would choose this approach:

  • NIC1 -> Hearbeat
  • NIC2 -> LAN(VLAN1), iSCSI(VLAN2)
  • NIC3 -> iSCSI(VLAN2)

Here you may see performance anomolies but it should be a stable configuration. I'm not sure if you are accessing your SAN from standalone servers, Hyper-V, or VMWare ESX. We have found out ESXi 5 will not load balance iSCSI correctly unless the 2 connections are in different vlans, so maybe in the latter scenario above you would have NIC3 -> iSCSI(VLAN3)

In any case, really investigate getting the multi-port nic cards if you can because with vlan and, in particular qos features, you have a much greater complexity in the network card driver and IP software stack. There can be bugs...

JGurtz
  • 523
  • 5
  • 13
1

All of this is possible doing exactly what you say using .1q, it's what it's designed for and works well. Can't add much more as you've got this tied up.

Chopper3
  • 101,299
  • 9
  • 108
  • 239
0

Why not to consider to move to 10G NICs for links you need to have increased bandwidth? The key idea your server isn't just for fun (or it is so?) so the more mess you create with VLANs, the more you (or your colleagues) will face later, increasing the risk of breaking the whole connectivity scheme.

10G NICs are more and more affordable these days, and you'll also will be able to use one 10G to carry traffic for several VLANs later if you need.

Alexander
  • 774
  • 2
  • 11
  • 20