I am in the process of re-doing the iSCSI network at my work. We currently have the following equipment:
- 1x Dell PowerConnect 6224 switch
- 1x Dell PowerVault MD3000 SAN connected to 2x Dell PowerEdge 1950 servers providing iSCSI
- 1x Dell PowerVault MD3000i SAN
- 2x Dell PowerEdge 2950 servers running ESX 3.5 soon to be ESX 4 - has 6 NICs
- 2x Dell ??? servers that have just been ordered for 2 more ESX hosts - has 8 NICs
All iSCSI traffic is on its own switch and is in the 192.168.1.x network. All other network traffic is on it's own network switch and is in the 10.10.x.x network. We have 2 NICs teamed (1 onboard Broadcom NIC and 1 Intel Pro 1000 NIC) to each ESX server in active/active state being connected to the single PC 6224 switch dedicated to iSCSI. All 4 NIC ports on the back of the MD3000i are connected to the same switch as well.
The problem with this setup is that the switch provides a large single point of failure. We are trying to correct/fix this with setting up a 2 switch network for iSCSI traffic for redundancy. I have 2 new PowerConnect 6224 switches that we will be using for this new network. The current switch that we have for iSCSI traffic will be then used for redundancy for the LAN side of the network or used as a vMotion network only between 4 ESX servers. (vMotion is currently a crossover connection between the 2 ESX servers
I have talked with Dell on a few occasions trying to figure this new network setup before we get the 2 new ESX servers that will be connecting to the MD3000i where our virtual machines are stored. I have come to the conclusion that it would be best to:
- Enable flowcontrol on switches - not currently setup
- Enable spanning-tree portfast on switches - not currently setup
- Setup jumbo frames on the switches, NICs and SAN - not currently setup
- Setup a 2 port LAG between the 2 switches
I am not sure that stacking the 2 PowerConnect switches is the best idea. Due to the fact that if the master switch were to fail the stack would reboot causing network outage while the stack re-elects a new master. Which to me sounds like it would not provide the redundancy/HA that we are looking for.
Since the MD3000i has 4 connections for the iSCSI traffic (2 for controller 0 and 2 for controller 1) connect the 0 side to switch A and then the 1 side to switch B. Then have a connection coming from our ESX servers to each switch for the iSCSI traffic.
My confusion about the setup comes with how the ESX server is configured. I am not sure how the 2 teamed NICs should be handled. From my understanding teamed NICs have to be connected to the same switch but we would be connecting them to 2 switches. Would we need to break the teaming and create a new vSwitch for each connection to switch A and B?
Is there a better way to configure this network or is the direction I am attempting to go the best?
Update: I am in the process of reading the iSCSI configuration guide for ESX 4. Will post back/mark answered once I finish reading that document.