Tuning Open-iSCSI initiator settings for VM LUNs

Question

I have a bunch of VM servers running Xen VMs from iSCSI LUNs mounted in the Dom0 host via Open-iSCSI. It's just a simple single-path setup (redundancy is on the target side via clustered storage and a virtual IP)

I'm considering using the settings from the Open-iSCSI README for iSCSI root, since that's basically what it is:

8.2 iSCSI settings for iSCSI root
---------------------------------

When accessing the root parition directly through a iSCSI disk, the
iSCSI timers should be set so that iSCSI layer has several chances to try
to re-establish a session and so that commands are not quickly requeued to
the SCSI layer. Basically you want theopposite of when using dm-multipath.

For this setup, you can turn off iSCSI pings by setting:

node.conn[0].timeo.noop_out_interval = 0 
node.conn[0].timeo.noop_out_timeout= 0

And you can turn the replacement_timer to a very long value:

node.session.timeo.replacement_timeout = 86400

Are there any other recommended settings for running VMs with a root on an iSCSI LUN I should think about?

Answer 1

First of all - what software/hardware you use for the target?

In my experience iSCSI initiators and targets usually don't require tuning, instead you must tune a network. Try connecting initiator and target using direct crossover cable, test a speed in this way, I bet it should be good. In this case the switch is the bottleneck. I can recommend:

• Move all iSCSI traffic to the separate VLAN
• Enable jumbo frames on the switch (also set MTU about 9000 on both initiator and target, be sure it's equal on both)
• Get a switch with a better pps
• Finally throw iSCSI away and move to the old good fibre channel. A $20 2Gbit FC card can give you about 200 Mbytes per sec, of course if your target storage can handle it.

Answer 2

I have found OpenISCSI/ietd to be problematic with virtualized environments. tgtd seems to be much more stable (no wonder it's the default target in RHEL)

just my 5c