I'm planning to run ZFS on Linux (latest Ubuntu stable ppa). The hardware is a xeon E3-1270 with 24 GB of RAM, 6 SAS ports on the motherboard, and a supermicro HBA with 8 SAS ports. I plan to have 4 x 3TB on 5400rpm SATA drives and 4 x 2TB on 7200rpm SATA drives in RAIDZ1 for a start. This will run 2 file servers in virtual machines. The file servers have different performance needs, so one will use the storage from the 5400rpm drives, and the other will use the 7200 rpm drives. The OS will run on a separate SSD.

The question is: how should I configure this (for performance, reliability, etc)?

a. Should I run ZFS on the VM host, and export the logical drive to the VM guest? Or should I export the raw disks to the VM guest and run ZFS on the guest?

b. For the file server with higher performance requirements, should I attach the 4 x 7200rpm drives to the HBA and attach the entire HBA to the VM guest?

  • Why bother separating or virtualizing the file servers? – ewwhite Feb 24 '13 at 6:00
  • 2
    Because we can? Though I'm not seeing a need to virtualize here. – Michael Hampton Feb 24 '13 at 6:05
  • Besides having different performance needs, the file servers also serve different groups of users. ie, one will be exposed to the internet via ftp,http,rsync,cvsup; the other is for the local network and mainly runs cifs. They're virtualized to enforce a clearer distinction in their trust boundaries. – Anon Feb 24 '13 at 6:06
  • Virtualization for that reason is extreme overkill... – Michael Hampton Feb 24 '13 at 6:09
  • I would also advise you to run the OS off of a thumb stick and use the SSD as an L2ARC. It can be shared between multiple pools and provide great read performance once it warms up. – RandomInsano Mar 21 '13 at 0:27

I don't see any need to virtualize anything in this instance. A file server doesn't need that level of separation... But of the options given, running ZFS natively on the host makes more sense. Don't bother with passthrough to your guests in this setup...

  • Thanks for the tip. Is the recommendation to reduce complexity (since you say not to bother with passthrough)? Or is running natively nearly equivalent or better for performance/reliability? – Anon Feb 24 '13 at 6:57
  • 1
    Reduce complexity and simplify management. If performance were the the most important consideration, you wouldn't be using virtual machines for this purpose. – ewwhite Feb 24 '13 at 7:00
  • I completely agree. If you need something that looks and smells like separate machines, look into LXC or chroot. Or FreeBSD jails/Solaris zones if you like learning new things. – RandomInsano Mar 21 '13 at 0:20

If you want performance forget RAIDZ1 (that seems to be a common knowledge among people using ZFS for a while). Instead use mirrors as your disk are not fast being high capacity and low rpm.

I would keep ZFS closer to hardware. There is no much penalty in giving virtual disks to VMs I would not run ZFS there unless there is a compelling reason (no raid in virtual servers).

As others said you should consider if you really need virtualization. My answer does not deal with that part.


For performance.. You would be best getting more drives, faster drives, (ie at least 7200 rpm) and/or getting ssds for caching - zil and l2arc. And/ or configuring as mirrors. You essentially will get the speed of two spindles, since you will have two vdevs. if instead you did say, 10 drives as mirrors, you will get 5 "spindles worth" of speed. You have 14 sas ports?

However its hard to answer without knowing for sure your needs. if one of the file servers is infrequently used, or used by only a few people, and say, mostly archival material, then speed is of no concern.


I'd also recommend you not to bother with putting host OS on SSD, but instead use SSD as cache device. ZFS really loves that.

P.S. I would also employ Proxmox VE as host server OS (good old stable Debian, really quite stable), which can use ZFS from the box.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.