I'm connecting a RHEL5 box to a Dell EqualLogic PS5000X and seeing a maximum of 1 Gbps even while using MPIO with three gigabit NICs.
I have all three interfaces on the PS5000X and three dedicated iSCSI interfaces on the server connected to the same switch. When using more than one interface to connect to the SAN I only get 1/n traffic on each. I followed the Dell setup guide and everything seems to be working perfectly aside from my throughput numbers.
[root@testbox ~]# multipath -ll linuxvol (36090a02840c3bd2f5ec324010000707f) dm-5 EQLOGIC,100E-00 [size=50G][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=3][enabled] \_ 6:0:0:0 sdf 8:80 [active][ready] \_ 14:0:0:0 sde 8:64 [active][ready] \_ 15:0:0:0 sdi 8:128 [active][ready]
[root@testbox ~]# dd if=/dev/zero of=/mnt/linuxvol/testfile obs=64k count=16M 16777216+0 records in 131072+0 records out 8589934592 bytes (8.6 GB) copied, 76.4894 seconds, 112 MB/s
I can even shut down iSCSI sessions mid-transfer and watch the interface utilization on the remaining connections jump from 33% to 50% to 100% as I go from three to two to a single session. I've tried tweaking the various queue depth and round robin settings which only make a small difference.
The PS5000X is set up with 14 data plus 2 hot spare 10k RPM SAS drives. My test volume is set to RAID-10 with multiple initiators enabled. No thin provisioning, no snapshots, no other traffic on the SAN.
What am I missing? Is there some MPIO setting I need to adjust on the OS?
Update: The switch is an HP ProCurve 4104GL. It doesn't support jumbo frames. Flow control is enabled on all of the SAN ports. Spanning tree is disabled. The only devices connected to the switch are the PS5000, the test server, and one uplink port. I know traffic isn't going through the uplink because I can watch port utilization on the switch while running the test. I'm also watching the network interface graph on the PS5000 and it lines up perfectly with the switch utilization, around 33% throughput on each interface.