The server has 4GB of RAM and my zpool is made up of 15.5k SAS drives arranged like this:

    tank          ONLINE       0     0     0
      raidz1-0    ONLINE       0     0     0
        c0t2d0    ONLINE       0     0     0
        c0t3d0    ONLINE       0     0     0
        c0t4d0    ONLINE       0     0     0
        c0t5d0    ONLINE       0     0     0
        c0t6d0    ONLINE       0     0     0
        c0t7d0    ONLINE       0     0     0
        c0t8d0    ONLINE       0     0     0
      raidz1-1    ONLINE       0     0     0
        c0t10d0   ONLINE       0     0     0
        c0t11d0   ONLINE       0     0     0
        c0t12d0   ONLINE       0     0     0
        c0t13d0   ONLINE       0     0     0
        c0t14d0   ONLINE       0     0     0
      c0t9d0      AVAIL
      c0t1d0      AVAIL   

The primary use is as an NFS store for a couple VMWare ESXi servers. I can't do any "true" benchmarks because this is a production system (no budget for test systems), but using dd and bonnie++ I can't get more than ~40-50MB/s writes and ~70-90MB/s reads. It seems I should be able to do much better, but I'm not sure where to optimize. Based on what I've read, I think dropping in a OCZ Vertex 2 Pro SSD as my L2ARC is going to be the best bang-for-the-buck to improve througput.

  1. Is there something else I should be looking into to help performance? If not...
  2. How do I know how big a cache device I need?
  3. Am I safe with only a single SSD as my cache device?
  • FWIW, bonnie++ showed similar I/O throughput both locally and over NFS. The big change was in the sequential/random create/stat/delete. NFS was slower by a longshot (8-10k/sec vs. 10-15/sec). – Sysadminicus Nov 10 '10 at 23:44
  • 1
    If you are trying to speed up writes, use a SLC SSD as a ZIL as well. Also add more RAM as that will with the L2ARC – tegbains Nov 11 '10 at 1:07
  • I came across this article at Nexenta with some details on how big the cache/log devices should be – Sysadminicus Nov 17 '10 at 18:09

My guess is that the ZFS Intent Log (ZIL) is limiting the write performance. A fast SSD used as a log device could help. Some hints:

  • ZIL Accelerator: DRAM or Flash? explains the issues in detail.
  • You may want to test with a disabled ZIL. See my other posting.
  • Checkout ZFS Features in Oracle Solaris Express if you're using a recent ZFS release because the way how to disable the ZIL has changed. There's a new sync filesystem property now.
  • Until recently it was impossible to remove a log device from a ZFS pool. Of course, this caused problems if the single log devices failed. It was therefore recommended to use a mirror for the log device. However, recent ZFS releases fixed this limitation and it's now possible to remove the single log device with zpool remove. Make sure you use a version that supports this feature if you really going to install the SSD.

I definitely would do it.. I have read that it helps substantially.. However you will need to get a quality SSD, as it will see A LOT of use, so the wear leveling should function well..

I found this today.. which would be very nice


  1. not sure
  2. 30 times your RAM should be enough.. ( 300GB plus should help buffer)
  3. The single SSD should be enough,its only a cache remember for throughput.. (Things will just get slow when it dies/breaks/doesnt exist)
  • One of the reasons folks were pointing to the Vertex 2 Pro was that it has a supercap to avoid data loss in the event of a power failure. Know if the Revodrive has one? – Sysadminicus Nov 10 '10 at 21:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.