ZFS is a modern file system and volume manager originally developed by Sun Microsystems and licensed under the CDDL. It is a copy-on-write file system with support for large storage arrays, protection against corruption, snapshots, clones, compression, deduplication and NFSv4 ACLs. An open-source ...

learn more… | top users | synonyms

144
votes
2answers
28k views

What are the different widely used RAID levels and when should I consider them?

This is a Canonical Question about RAID levels. What are: the RAID levels typically used (including the RAID-Z family)? deployments are they commonly found in? benefits and pitfalls of each?
43
votes
12answers
36k views

ZFS vs XFS

We're considering building a ~16TB storage server. At the moment, we're considering both ZFS and XFS as filesystem. What are the advantages, disadvantages? What do we have to look for? Is there a ...
38
votes
1answer
2k views

I am going to chop the FreeBSD automounter into little pieces and boil them in oil

I am trying to expose a hierarchy of home directories to a number of FreeBSD jails. The home directories are configured such that each is a unique ZFS dataset. The jails are used for development ...
35
votes
5answers
8k views

ZFS on FreeBSD: recovery from data corruption

I have several TBs of very valuable personal data in a zpool which I can not access due to data corruption. The pool was originally set up back in 2009 or so on a FreeBSD 7.2 system running inside a ...
26
votes
1answer
3k views

ZFS endless resilvering

I have a large (> 100TB) ZFS (FUSE) pool on Debian that lost two drives. As the drives failed, I replaced them with spares until I could schedule an outage and physically replace the bad disks. When ...
26
votes
7answers
16k views

Is bit rot on hard drives a real problem? What can be done about it?

A friend is talking with me about the problem of bit rot - bits on drives randomly flipping, corrupting data. Incredibly rare, but with enough time it could be a problem, and it's impossible to detect....
24
votes
4answers
7k views

LVM snapshots vs. file system snapshots

As far as I know, LVM makes it possible to take snapshots of a volume. There are also a number of file systems (ZFS, Btrfs, reiserfs, ...) which supports snapshots. However, I've never understood the ...
23
votes
2answers
2k views

Why are all the zpools named “tank”?

In practically every example of ZFS usage that I've seen online (including several questions here), the zpool is named "tank". Why? Is there some sort of significance to the name or is it just that ...
21
votes
2answers
5k views

ZFS Data Loss Scenarios

I'm looking toward building a largish ZFS Pool (150TB+), and I'd like to hear people experiences about data loss scenarios due to failed hardware, in particular, distinguishing between instances where ...
20
votes
1answer
15k views

Hosting a ZFS server as a virtual guest

I'm still new to ZFS. I've been using Nexenta but I'm thinking of switching to OpenIndiana or Solaris 11 Express. Right now, I'm at a point of considering virtualizing the ZFS server as a guest within ...
20
votes
2answers
789 views

Sanity check on 40TB server configuration

I've got 40 years in computing, but I've never had to build a server quite like this one, so this might be a n00b question. I have a client that is going to offer ultra-high def music files for ...
19
votes
6answers
13k views

How does BTRFS compare to ZFS?

I am considering which OS and filesystem to use on some new servers I have and am considering either Free BSD with ZFS, or Linux with BTRFS. The programs I have run on both systems, so the only ...
16
votes
5answers
12k views

ZFS best practices with hardware RAID

If one happens to have some server-grade hardware at ones disposal, is it ever advisable to run ZFS on top of a hardware-based RAID1 or some such? Should one turn off the hardware-based RAID, and run ...
16
votes
3answers
3k views

150 TB and growing, but how to grow?

My group currently has two largish storage servers, both NAS running debian linux. The first is an all-in-one 24-disk (SATA) server that is several years old. We have two hardware RAIDS set up on it ...
15
votes
2answers
27k views

How can I add one disk to an existing raidz zpool?

I have an OpenSolaris server with a zpool backupz comprised of four SCSI drives: -bash-3.2# zpool status backupz pool: backupz state: ONLINE scrub: none requested config: NAME ...
15
votes
5answers
16k views

How to delete all but last [n] ZFS snapshots?

I'm currently snapshotting my ZFS-based NAS nightly and weekly, a process that has saved my ass a few times. However, while the creation of the snapshot is automatic (from cron), the deletion of old ...
15
votes
4answers
6k views

ZFS and SAN — best practices?

Most discussions of ZFS suggest that the hardware RAID be turned off and that ZFS should directly talk to the disks and manage the RAID on the host (instead of the RAID controller). This makes sense ...
15
votes
7answers
3k views

recommendations for efficient offsite remote backup solution of vm's

I am looking for recommendations for backing up my current 6 vm's(and soon to grow to up to 20). Currently I am running a two node proxmox cluster(which is a debian base using kvm for virtualization ...
14
votes
3answers
6k views

ZFS RAID and LUKS encryption in Linux

I'm planning to setup a set of 3x 2TB 7200rpm drives as a LUKS-encrypted Z-RAID pool in Linux (for a NAS solution). My understanding of the problem at hand is that the only way to achieve this is to ...
14
votes
2answers
17k views

Transparent compression filesystem in conjunction with ext4

I am trying to test a project that needs compressed storage with use of the ext4 file system since the application I use relies on ext4 features. Are there any production/stable solutions out there ...
13
votes
6answers
29k views

ZFS SAS/SATA controller recommendations

I've been working with OpenSolaris and ZFS for 6 months, primarily on a Sun Fire x4540 and standard Dell and HP hardware. One downside to standard Perc and HP Smart Array controllers is that they do ...
13
votes
12answers
9k views

Best compression for ZFS send/recv

I'm sending incremental ZFS snapshots over a point-to-point T1 line and we're to a point where a day's worth of snapshots can barely make it over the wire before the next backup starts. Our send/recv ...
13
votes
2answers
13k views

For L2ARC and ZIL: is it better to have one large SSD for both, or two smaller SSDs?

Main references ZFS L2ARC (Brendan Gregg) (2008-07-22) and ZFS and the Hybrid Storage Concept (Anatol Studler's Blog) (2008-11-11) include the following diagram: Question Should I interpret the ...
13
votes
1answer
751 views

Force update checksum on zfs?

I recently changed the checksum property on one of my non-duplicated zfs filesystems to sha256 from on (fletcher4) to better support the sending of duplicated replication steams, as in this command ...
13
votes
1answer
2k views

Why did rebooting cause one side of my ZFS mirror to become UNAVAIL?

I just recently migrated a bulk data storage pool (ZFS On Linux 0.6.2, Debian Wheezy) from a single-device vdev configuration to a two-way mirror vdev configuration. The previous pool configuration ...
12
votes
3answers
7k views

Non-ECC memory with ZFS: a stupid idea?

I have a new server and am planning to upgrade the paltry 2 GB of memory to the maximum of 16 GB. (Theoretically 8 GB is the limit, but empirically 16 GB has been shown to work.) Some guides advise ...
12
votes
7answers
8k views

ZFS alternative for Linux?

I'm running OpenSolaris with ZFS for my main fileserver. I originally went with ZFS because I heard so many awesome things about it: Automatic disk spanning (zpools) Software RAID (RAID-Z) Automatic ...
12
votes
2answers
8k views

ZFS performance: do I need to keep free space in a pool or a file system?

I know that the performance of ZFS heavily depends on the amount of free space: Keep pool space under 80% utilization to maintain pool performance. Currently, pool performance can degrade when a ...
12
votes
1answer
499 views

Technical details for Server 2012 de-duplication feature

Now that Windows Server 2012 comes with de-duplication features for NTFS volumes I am having a hard time finding technical details about it. I can deduce from the TechNet documentation that the de-...
11
votes
2answers
16k views

ZFS: Mirror vs. RAID-Z

I'm planning on building a file server using OpenSolaris and ZFS that will provide two primary services - be an iSCSI target for XenServer virtual machines & be a general home file server. The ...
11
votes
5answers
7k views

Which is faster for read access on EC2; local drive or EBS?

Which is faster for read access on an EC2 instance; the "local" drive or an attached EBS volume? I have some data that needs to be persisted so have placed this on an EBS volume. I'm using ...
11
votes
3answers
2k views

ZFS: How do you restore the correct number of copies after losing a drive?

With zfs, if you have copies=2 and then you lose a drive containing some of those copies, how do you tell the system that it should make a new copy of the data blocks for the affected files? Or does ...
11
votes
1answer
8k views

Is calculating IOPS for ZFS RAIDZ different then calculating IOPS for RAID5 & RAID6?

When calculating IOPS for traditional RAID arrays, one could use the following formula (Borrowed from Getting The Hang Of IOPS v1.3 on Symantec Connect): Ieffective = (n * Isingle) / (READ% + (F * ...
11
votes
3answers
8k views

Tuning ZFS scrubbing, 141KB/s running for 15 days

A pretty basic system running mirror+stripe on 7.2k rpm sas disks, not particularly loaded. No dedup, compression on all datasets. Scrub has been running for 15 days at the speed of a dead snail. Is ...
11
votes
2answers
948 views

Why are all of my ZFS snapshot directories empty?

I'm running an Oracle 11 box as a ZFS storage appliance, and I'm taking regular snapshots of the ZFS filesystems, via cron. In the past, I know that if I wanted to grab a particular file from a ...
10
votes
2answers
8k views

How to upgrade a ZFS RAID-Z array to larger disks on OpenSolaris?

I'm planning on installing OpenSolaris on my home server (right now it has Linux) and I would like to know how to prepare for upgrading the server to have larger hard drives in the future. Now the ...
10
votes
4answers
6k views

Can I trust that ZFS is consistent between Linux and FreeBSD?

I'm planning to build a FreeNAS box sometime soon, but if ZFS on Linux eventually proves to be reliable, I might want to switch, just to have a more familiar OS. So I'm wondering if I can trust that ...
10
votes
3answers
10k views

Is a large RAID-Z array just as bad as a large RAID-5 array?

For a long time I've heard about how bad an idea a large (>5TB?) RAID-5 array is simply because there's a high risk for another drive to fail. Has RAID-Z1 managed to remedy this for an array of any ...
10
votes
2answers
2k views

Are consumer class hard disks okay for zfs?

I just recently bought a new server an HP DL380 G6. I replaced the stock smart array p410 controller with an LSI 9211-8i. My plan is use ZFS as the underlying storage for XEN which will run on the ...
10
votes
5answers
7k views

Best filesystem choices for NFS storing VMware disk images

Currently we use an iSCSI SAN as storage for several VMware ESXi servers. I am investigating the use of an NFS target on a Linux server for additional virtual machines. I am also open to the idea of ...
10
votes
3answers
5k views

ZFS - Impact of L2ARC cache device failure (Nexenta)

I have an HP ProLiant DL380 G7 server running as a NexentaStor storage unit. The server has 36GB RAM, 2 LSI 9211-8i SAS controllers (no SAS expanders), 2 SAS system drives, 12 SAS data drives, a hot-...
10
votes
1answer
6k views

How large is my ZFS dedupe table at the moment?

I have read a lot of information about planning RAM requirements forZFS deduplication. I've just upgraded my file server's RAM to support some very limited dedupe on ZFS zvols which I cannot use ...
10
votes
5answers
4k views

ZFS: Redistribute zvol over all disks in the zpool?

Is there a way in which ZFS can be prompted to redistribute a given filesystem over the all of the disks in its zpool? I'm thinking of a scenario where I have a fixed size ZFS volume that's exported ...
10
votes
1answer
1k views

Identifying the number of changed bytes between two ZFS snapshots of the same file

Let's assume I have a ZFS filesystem that contains virtual machine disk images, e.g. /tank/examplevm/examplevm-flat.vmdk Let's further assume I take daily snapshots of that ZFS filesystem, e.g. $ ...
10
votes
3answers
13k views

Are ZFS clustered filesystems possible?

Is it possible to create a ZFS cluster? Or would you need to go the seemingly ugly (to me at least) route of ZFS with UFS formatted pools governed by GlusterFS? The idea is to see if it is possible ...
9
votes
6answers
10k views

Recommendations on ZFS on FreeBSD as a NAS box?

Please note that the answers on this page are from 2009, and should not be relied on as gospel. If you have a specific question about zfs then please click the Ask a Question button and ask a specific ...
9
votes
2answers
14k views

Need to replace disk in zpool … confused

I need to replace a bad disk in a zpool on FreeNAS. zpool status shows pool: raid-5x3 state: ONLINE scrub: scrub completed after 15h52m with 0 errors on Sun Mar 30 13:52:46 2014 config: ...
9
votes
4answers
17k views

Moving a ZFS filesystem from one pool to another

I am extending a Solaris 10 U8 server with an extra storage pool in addition to the root rpool. What is the most effective / simple / reliable way of moving particular ZFS filesystem(s) from rpool to ...
9
votes
6answers
3k views

File size with zfs compression

I usually estimate the size of a whole directory tree using du -ks $DIRECTOY_TREE_ROOT, but this method cannot be used when zfs compression is on. The total displayed by ls -l is ok for a single ...
9
votes
2answers
3k views

ZFS and cache devices

I'm going to add a SSD drives to my ZFS storages as a cache. Anyone has any experience with this? What if cache device dies, will the any data be affected?