10

I'm going to add a SSD drives to my ZFS storages as a cache. Anyone has any experience with this? What if cache device dies, will the any data be affected?

2 Answers 2

7

http://web.archive.org/web/20100911224754/http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide

(The Solaris Internals website no longer up but WebArchive has a copy)

0
14

Basically there's three types of ZFS cache, all used for both data and metadata.

  • ARC (Adaptive Replacement Cache) - Main memory DRAM cache for reads and writes.
  • L2ARC (Level 2 ARC) - safe read cache: no data loss/service interruption from device failure. Usually SSD based.
  • ZIL (ZFS Intent Log) - safely holds writes on permanent storage which are also waiting in ARC to be flushed to disk. Data should rarely live in this cache for longer than 30secs and data is never read except after a crash to replay any uncommitted pool writes. On recent any recent ZFS version, Zil device failure won't cause data loss (all data still in ARC), but device failure + a crash or power outage may cause some writes to get lost.

Upgrade your ARC first, buy oodles of main memory. Note L2ARC and Zil both have overhead allocated out of the ARC too.

L2Arc is populated by read-cached blocks as they are evicted from ARC. ZFS by default only caches random IO (small reads) into L2ARC and is not used for streaming workloads (unless instructed to). You can basically use any device for this (including a fast 15k HD) but it works best with an SSD that handles many random read IOPS with ease.

ZIL accelerates workloads which require synchronous writes (processes wait for confirmation that writes have actually been committed to disk before continuing execution). Zil performs a similar role to battery backed cache on high end RAID controllers. Although write latency and streaming write IOPS are what define a good Zil SSD, a Zil above all else mustn't ever loose any data in the event power loss. Many suitable devices have a super-capacitor to finalize any pending operations without system power. SLC SSDs with high write endurance (Intel X25-E) used to be recommended, but newer devices use RAM with battery/supercap to write back to NAND in the event of a power failure. ZILs need not be large, but by using only a small fraction of a large device (e.g. 8GB out of a 300GB Intel 320 MLC SSD) you can yield much higher effective write endurance. 'Enterprise' vendors always recommend mirrored ZILs, my workloads have never been that important.

As for specific products, STEC made the first SSDs for Sun's Fishworks project (both Logzilla & Readzilla) and has current devices for both ZIL (ZeusRAM $2500/8GB) and L2ARC (Zeus IOPS $3k/400GB) which both come highly recommended. PCIe based SSDs are also worth considering, like the ZIL-specific DDRdrive x1 ($2k/4GB) or any big PCIe SSD for L2ARC. Other less performant (read:cheaper) 2.5inch SSD devices can also offer significant performance gains especially when used in aggregate for L2ARC.

3
  • u sure OCZ Vertex2 is ddr based? I see it's nand-mlc based
    – disserman
    Aug 15, 2010 at 21:19
  • 3
    OCZ Vertex2Pro is MLC NAND, but unlike almost every other SSD it has a supercap allowing writes in progress to complete when power is pulled. DDRdrive X1 is 4GB DDR2, 4GB SLC and a supercap that copies ram to NAND in 60secs when power is lost. Logzilla is write optimized SLC NAND with a supercap (no DDR either)
    – notpeter
    Aug 15, 2010 at 22:03
  • thanks, I have 2 kingston (intel-based) ssds which I gonna use as a cache devices until they die or warranty will be over. then look at your suggestions.
    – disserman
    Aug 16, 2010 at 0:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .