ZFS and cache devices

Question

I'm going to add a SSD drives to my ZFS storages as a cache. Anyone has any experience with this? What if cache device dies, will the any data be affected?

Answer 1

http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide#Separate_Cache_Devices

Answer 2

Basically there's three types of ZFS cache, all used for both data and metadata.

• ARC (Adaptive Resource Cache) - Main memory DRAM cache for reads and writes.
• L2ARC (Level 2 ARC) - safe read cache: no data loss/service interruption from device failure. Usually SSD based.
• ZIL (ZFS Intent Log) - safely holds writes on permanent storage which are also waiting in ARC to be flushed to disk. Data should rarely live in this cache for longer than 30secs and data is never read except after a crash to replay writes that weren't committed to the pool. On recent OpenSolaris/Nexenta, device failure won't cause data loss (the data is still in ARC), but device failure + a crash or power outage may cause data loss.

Upgrade your ARC first, an extra 4-8GB is definitely worth it. Both L2ARC and Zil require memory in ARC to operate.

L2Arc is populated by read-cached blocks as they are evicted from ARC. It only caches small reads and is not used for streaming workloads. You can basically use any device for this (including a fast HD) but it works best with an SSD that can handle high IOPS load like the Intel X25-M 160GB ($415), but if you've got deep pockets consider one or multiple Sun Readzilla 100GB ($5k) or Intel X25-E 64GB ($700) devices for increased performance.

ZIL accelerates workloads which require synchronous writes (processes wait for confirmation that writes have actually been committed to disk before continuing execution). Zil performs a similar role to battery backed cache on high end RAID controllers. Since not a single write can be lost when power fails, usually these devices have a super-capacitor . The Sun Logzilla ($6k), DDRdrive X1 ($2k) and OCZ Vertex2Pro ($425) are basically the only devices well suited for this.