We have an iSCSI SAN that will be used as backing store for virtual machines. We use Xen and will directly present the LUNs as block devices of VMs (so there is no file system with files on it for each VM, like VMware does when using VMFS datastores).

I'm wondering if it would be better to provision separate LUNs for operating system and data of each vm (eg. one small lun to install Linux and one bigger lun to keep mailboxes, one small for linux and one big for webserver data, and so on...) or just one big LUN per vm.

Looking at the former, then one could ask "so why not even another one LUN for the swap partition", going on until you just have only one "partition" on each lun.

Keep in mind all these LUNs are from the exact same RAID volume on the SAN so there's no point in differentiating for disk speed or raid level.

closed as primarily opinion-based by Bryan, mdpc, Scott Pack, kce, Falcon Momot Jul 26 '13 at 22:57

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • This is more of an opinion question, and for every 10 people you ask, you'll get 10 different and mostly mutually contradictory answers. – John Jul 26 '13 at 17:46

Right now you may not plan on having different RAID levels or disk speeds. But down the road, when your mailboxes grow to multiple TB, or you start archiving large video files, or what have you, you may need to.

Ignoring the SAN aspect of it, this boils down the same as having one hard drive with partitions (single partitioned LUN), or two separate hard drives (2 LUNs).

Since with the SAN it costs you almost nothing to provide the disk as seperate LUNs, I would seperate them. Keeping your data independent of the OS means you can do fun things like:

  • Seperate raid/disk speed/caching levels - some large capacity disks currently only come in slower RPM models, so you need to split it up if you need high storage capacity
  • More easily split the data between multiple SANs if needed
  • Upgrade/repair your OS by installing to a fresh virtual machine, and simply remapping the data LUN to the new server when you're ready. No long copy process, little downtime.
  • Possibly faster SAN snapshots, if you use them

The downsides:

  • Management overhead, having to manage multiple LUNs for each VM
  • Some SANs have only have a license for so many LUNs, so you could run out on a lower licensed SAN
  • Separate LUNs also gives you separate IO queues per LUN. Desirable! – MikeyB Jul 26 '13 at 17:49
  • Having used both this scheme and the VMFS method I highly prefer the partition=LUN method. It's just more flexible. Do bear in mind the licensing issue. The amount of RAM/CPU power the NAS can provide might also be a concern. Each additional LUN causes RAM and CPU overhead at NAS level. This can get out of hand real quickly. – Tonny Jul 26 '13 at 18:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.