I'll be using this iSCSI target for a couple of Debian-based KVM virtual machine hosts. Each of the target's redundant controllers has 4 ethernet ports; likewise for the initiators. I use two switches (ZyXEL GS-2200-24) with a trunk between them, and VLANs isolate each path. I've also enabled jumbo frames and flow control.
The MPIO system in this Debian release is fabulous: as long as dm-multipath is loaded before logging in to the iSCSI target, everything Just WorksTM without any configuration file, provided I load scsi_dh_rdac before-hand.
That's the first snag: I can change some of the defaults if I provide an
/etc/multipath.conf file. I tested with
use_friendly_names yes, which successfully creates an
mpath0 link in
/dev/mapper/ – instead of using the unfriendly-looking WWID. But if I try to change
rr_min_io from the default 1000 down to 8, I get ignored; so I do this pretty dance:
dmsetup suspend mpath0 dmsetup table mpath0 | sed 's, 1000, 8,g' | dmsetup reload mpath0 dmsetup resume mpath0
That changes the number of requests sent down one of the quad links before round-robin kicks in and sends it down the next, from the default 1000 down to 8. This actually changes the multipath table (as per
multipath -v3 | grep params). How does one configure this default in the new multipath code? I'm assuming this worked before multipath went all dynamic and self-configuring... At least all vendor docs I've read, and other discussions on the web kind of assume this worked.
A simple sequential write using
dd bs=100M count=50 if=/dev/zero of=/dev/mapper/mpath0-part1 & sync goes from ~135MB/s up to ~260MB/s with this change. And that's the second snag: that's about 2Gbps instead of the 4Gbps I actually have between the initiator and the target. Running
iostat -kd 1 for 1sec running updates shows only 2 of the 4 paths being filled up.
This LUN is short-stroked: its 16GB reside right at the beginning of a 12-spindle RAID10 array of 600 6Gbps SAS disks spinning at 15,000rpm. I was expecting this to be enough to saturate the 4Gbps I have; am I correct?