4

I have an IBM x3650 with the following components:

  • 6 Gb NICs (all connected to an Extreme Networks x450 switch that also houses the ISCSI vlan and the SAN ISCSI connections)

I need to configure it properly for software ISCSI to connect to a LUN on a SAN.

Currently I have it setup as follows AND IT DOES connect and see the LUN:

  • 1 NIC - vSwitch0 / vswif0
  • 2 NICs - vSwitch1 / vswif1/vmkernel

My question really is:

A) What exactly should I do with all 6 nics to utilize them properly for redundancy and load balancing? B) Can I gain throughput (teaming) on vmkernel for the ISCSI operations?

I'm going to be adding a 2nd identical server once I have the right configuration on this one to the same LUNs so that I can do vmotion etc.

Any guidance is appreciated. Ask questions if you need clarification. A few screenshots are attached as well

VERSION is 3.5.0, 158874

enter image description here

TheCleaner
  • 32,627
  • 26
  • 132
  • 191

2 Answers2

4

This is the ideal config for 6 nics assuming you will add addtinal servers and utilize HA, DRS and vMotions.

Nic1 Service Console in main vlan100, main subnet

Nic2 Vmotion in main vlan100 Separate subnet

Nic3 and Nic4 both trunk ports for VMs to use

Nic5 and Nic6 - iSCSI Nics – in iscsi vlan, 2 separate subnets (assuming your SAN/NAS as 2, dual port controllers.

Be advised it is NOT recommended that you use teaming with your iSCSI nics. They should also be separate from your other services, I.E. vMotion, SC, Trunks etc. Using teaming will cause a delay in the event of a failure. It can also slow down the systems as your storage switches between controllers. I can explain in more detail if needed.

The vNIics however should use team. The nics in that team, Nic3 and Nic4, should be trunk ports on your switch.

Alan
  • 856
  • 1
  • 9
  • 18
  • 1
    I am quite interested into why teaming the iSCSI NICs is not recommended. Can you please elaborate? – Mike Fiedler Jun 03 '09 at 16:24
  • According to the VI3 iSCSI design guide, you can use NIC teaming, you just have to disable port security and is used for failover only, not multipathing. "If you plan to use NIC teaming to increase the availability of your network access to the iSCSI storage array, you must turn off port security on the switch for the two ports on which the virtual IP address is shared." – Kevin Kuphal Jun 03 '09 at 16:31
  • Link to the design guide: http://www.vmware.com/files/pdf/iSCSI_design_deploy.pdf – Kevin Kuphal Jun 03 '09 at 16:32
1

One thing to consider is that even with teaming, your throughput will be limited to the IP source/destination combination. That is, if you have an ESX host talking to an iSCSI host, their traffic will only utilize one Gbps port of the team. You should consider, on the iSCSI side, having multiple IP addresses that ESX can connect to which can better spread the load.

That being said, I would probably use it in his configuration:

  • Team 1: 3 NICs: Use for service console, vmotion, and vmkernel (iSCSI) operations
  • Team 2: 3 NICs: Use for VM networking

This gives you redundancy on each piece (service console, vmotion, vmkernel, vm networking) without carving up your NICs too much.

Kevin Kuphal
  • 9,134
  • 1
  • 35
  • 41
  • Thanks Kevin. There are 2 IPs for the LUNs as shown in the pic (and 2 NICs connected to the SAN for it). So when adding the 3 Nics per team, how do I add it and set the redundancy in VMWARE? I know how to do it in Windows but it seems like I don't actually set anything other than my fault tolerance settings (is the first choice the typical setting?) – TheCleaner Jun 03 '09 at 16:05
  • I don't believe you have to do anything special on the VMware side for the team. VMware does not use LACP so you will have to mark your switches as a team (not sure what kind of switches you have). As for the iSCSI config, I don't use it here, but as I understand it you add each IP for your SAN as an iSCSI HBA and then you can set the path for each LUN to which HBA you want. – Kevin Kuphal Jun 03 '09 at 16:37