Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an IBM x3650 with the following components:

  • 6 Gb NICs (all connected to an Extreme Networks x450 switch that also houses the ISCSI vlan and the SAN ISCSI connections)

I need to configure it properly for software ISCSI to connect to a LUN on a SAN.

Currently I have it setup as follows AND IT DOES connect and see the LUN:

  • 1 NIC - vSwitch0 / vswif0
  • 2 NICs - vSwitch1 / vswif1/vmkernel

My question really is:

A) What exactly should I do with all 6 nics to utilize them properly for redundancy and load balancing? B) Can I gain throughput (teaming) on vmkernel for the ISCSI operations?

I'm going to be adding a 2nd identical server once I have the right configuration on this one to the same LUNs so that I can do vmotion etc.

Any guidance is appreciated. Ask questions if you need clarification. A few screenshots are attached as well

VERSION is 3.5.0, 158874

http://i44.tinypic.com/161c5xh.jpg

share|improve this question
    
What kind of switches are you using? –  andyhky Jun 3 '09 at 16:11
    
It's on the first bullet point. Extreme Networks x450. –  TheCleaner Jun 4 '09 at 13:03
add comment

2 Answers 2

up vote 3 down vote accepted

This is the ideal config for 6 nics assuming you will add addtinal servers and utilize HA, DRS and vMotions.

Nic1 Service Console in main vlan100, main subnet

Nic2 Vmotion in main vlan100 Separate subnet

Nic3 and Nic4 both trunk ports for VMs to use

Nic5 and Nic6 - iSCSI Nics – in iscsi vlan, 2 separate subnets (assuming your SAN/NAS as 2, dual port controllers.

Be advised it is NOT recommended that you use teaming with your iSCSI nics. They should also be separate from your other services, I.E. vMotion, SC, Trunks etc. Using teaming will cause a delay in the event of a failure. It can also slow down the systems as your storage switches between controllers. I can explain in more detail if needed.

The vNIics however should use team. The nics in that team, Nic3 and Nic4, should be trunk ports on your switch.

share|improve this answer
1  
I am quite interested into why teaming the iSCSI NICs is not recommended. Can you please elaborate? –  Mike Fiedler Jun 3 '09 at 16:24
    
According to the VI3 iSCSI design guide, you can use NIC teaming, you just have to disable port security and is used for failover only, not multipathing. "If you plan to use NIC teaming to increase the availability of your network access to the iSCSI storage array, you must turn off port security on the switch for the two ports on which the virtual IP address is shared." –  Kevin Kuphal Jun 3 '09 at 16:31
    
Link to the design guide: vmware.com/files/pdf/iSCSI_design_deploy.pdf –  Kevin Kuphal Jun 3 '09 at 16:32
add comment

One thing to consider is that even with teaming, your throughput will be limited to the IP source/destination combination. That is, if you have an ESX host talking to an iSCSI host, their traffic will only utilize one Gbps port of the team. You should consider, on the iSCSI side, having multiple IP addresses that ESX can connect to which can better spread the load.

That being said, I would probably use it in his configuration:

  • Team 1: 3 NICs: Use for service console, vmotion, and vmkernel (iSCSI) operations
  • Team 2: 3 NICs: Use for VM networking

This gives you redundancy on each piece (service console, vmotion, vmkernel, vm networking) without carving up your NICs too much.

share|improve this answer
    
Thanks Kevin. There are 2 IPs for the LUNs as shown in the pic (and 2 NICs connected to the SAN for it). So when adding the 3 Nics per team, how do I add it and set the redundancy in VMWARE? I know how to do it in Windows but it seems like I don't actually set anything other than my fault tolerance settings (is the first choice the typical setting?) –  TheCleaner Jun 3 '09 at 16:05
    
I don't believe you have to do anything special on the VMware side for the team. VMware does not use LACP so you will have to mark your switches as a team (not sure what kind of switches you have). As for the iSCSI config, I don't use it here, but as I understand it you add each IP for your SAN as an iSCSI HBA and then you can set the path for each LUN to which HBA you want. –  Kevin Kuphal Jun 3 '09 at 16:37
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.