I have a bunch of VM servers running Xen VMs from iSCSI LUNs mounted in the Dom0 host via Open-iSCSI. It's just a simple single-path setup (redundancy is on the target side via clustered storage and a virtual IP)

I'm considering using the settings from the Open-iSCSI README for iSCSI root, since that's basically what it is:

8.2 iSCSI settings for iSCSI root

When accessing the root parition directly through a iSCSI disk, the
iSCSI timers should be set so that iSCSI layer has several chances to try
to re-establish a session and so that commands are not quickly requeued to
the SCSI layer. Basically you want theopposite of when using dm-multipath.

For this setup, you can turn off iSCSI pings by setting:

node.conn[0].timeo.noop_out_interval = 0 
node.conn[0].timeo.noop_out_timeout= 0

And you can turn the replacement_timer to a very long value:

node.session.timeo.replacement_timeout = 86400

Are there any other recommended settings for running VMs with a root on an iSCSI LUN I should think about?

  • 1
    Its so much better to run initiators in the dom-u .. is this what you are doing? Dom-0 has limited elbow room, fire those within the guest itself. – Tim Post Oct 7 '09 at 18:49
  • Also keep in mind that the iSCSI control plane remains entirely in userspace ..keep those initiators within the guest itself :) – Tim Post Oct 7 '09 at 18:51

First of all - what software/hardware you use for the target?

In my experience iSCSI initiators and targets usually don't require tuning, instead you must tune a network. Try connecting initiator and target using direct crossover cable, test a speed in this way, I bet it should be good. In this case the switch is the bottleneck. I can recommend:

  • Move all iSCSI traffic to the separate VLAN
  • Enable jumbo frames on the switch (also set MTU about 9000 on both initiator and target, be sure it's equal on both)
  • Get a switch with a better pps
  • Finally throw iSCSI away and move to the old good fibre channel. A $20 2Gbit FC card can give you about 200 Mbytes per sec, of course if your target storage can handle it.
| improve this answer | |

I have found OpenISCSI/ietd to be problematic with virtualized environments. tgtd seems to be much more stable (no wonder it's the default target in RHEL)

just my 5c

| improve this answer | |
  • Not relevant here, I'm talking about the initiator side. I will clarify the question. – Kamil Kisiel Oct 9 '09 at 16:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.