I'm going to build a ZFS file server from FreeBSD. I learned recently that I can't expand a RAIDZ udev once it's part of the pool. That's a problem since I'm a home user and will probably add one disk a year tops.

But what if I set copies=3 against my entire pool and just throw individual drives into the pool separated? I've read somewheres that the copies will try and distribute across drives if possible. Is there a guarantee there? I really just want protection from bit rot and drive failure on the cheap. Speed's not an issue since it'll go over a 1Gb network and at MOST stream 720p podcasts.

Would my data be guaranteed safe from a single drive failure? Are there things I'm not considering? Any and all input is appreciated.

  • 1
    Huh. Big issue might be lack of re-silvering eh? Like, if a drive with one copy of the data fails, will 'zpool scrub' create any missing duplicates? – RandomInsano Dec 29 '09 at 7:46
  • I think I need to learn more about RAID5, since if I set copies=2 I would effectively create a mirror and halve my storage, whereas RAIDZ/5 would take only one complete drive's worth of space away. I'm a tad confused. – RandomInsano Dec 29 '09 at 9:23
  • You mentioned dedup in the subject line... just FYI, dedup doesn't play nicely with snapshots. I'd recommend against using dedup unless you really know a lot about your data and how you're going to be using it. – jlp May 7 '12 at 21:02
  • I just caught myself misreading it to, I actually wrote 'duplication'. I've renamed it with the right wording. May no-one else misread this again! :D – RandomInsano Jun 10 '12 at 8:16

You will not be protected using a raidz vdev + single disk vdevs in the same pool. You should be adding two disks at the time (and mirror them) to your zpool, that will cover one disk failure.

The reason is that copies does not guarantee that the blocks will end up on different disks.

One alternative to expanding your raidz vdev is to use zfs send to store all you data somewhere temporarily while you add a disk and rebuild your raidz vdev and then zfs receive to get it back. It will be hard once you get past a few TB as you need a lot of space to store your data.

  • Even if copies make sure the block end up on different disk (which it basically does), the whole pool would be doomed in case of whole disk failure. – jlliagre May 11 '12 at 23:54

The file system data would very likely be redundant, especially with copies=3 so your data would be protected against bit rot.

However, it won't be reliably protected against a full disk failure as there would be no way to replace the faulty disk by a new one. Even while just after the failure your should be able to access all your data, your system won't survive a reboot as your pool would not be importable.

See ZFS: How do you restore the correct number of copies after losing a drive? for a similar question.

  • Because of the faulted state, or because of data being missing? – RandomInsano May 8 '12 at 18:56
  • 1
    No data should be missing in your specific case but the pool is unaware of that. The zpool command would refuse to re-import your pool because of the faulted state. The only solution should you lose a disk would be to quickly backup your data elsewhere, destroy and recreate the pool then restore from backup. – jlliagre May 8 '12 at 20:37

Interesting, found this today. Copies should guarantee ditto blocks created by 'copies' are stored on different vdevs: https://blogs.oracle.com/bill/entry/ditto_blocks_the_amazing_tape (see section "Spread 'em")

So, according to this blog post, you would be protected against full disk failures on file systems which have multiple copies available. But only on those file systems.

"In a storage pool with multiple devices (vdevs), things get a little spicier. We allocate each copy of a block on a separate vdev"

  • 1
    You are missing the point dataset only exist on top of pools and if the pool is not redundant, file system redundancy is ephemeral. – jlliagre May 7 '12 at 7:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.