2

I have never encountered a disk failure in my whole life but I had a dozen of bit rot and silent corruptions. 'Normal' file systems and hardware RAID can do absolutely nothing about it. I have been using application-level parity redundancy to protect my most important data, but it's apparently not quite effective.

I know ZFS store checksum of the chunks in software RAID, which offers protection against bit rot and silent corruption. The problem is, I do not want RAID and that's off the table.

So is it possible to use filesystem-level parity without RAID or multi-disk systems ? For example by allocation part of the disk space to store the parity of the "chunks" instead of storing the parity of equal sized disks on another disk as in RAID5.

Thanks everyone for answering.The following is the conclusion based on the answers:

No, it's not possible to use ZFS parity without RAID, but it's possible to use mirror copy without RAID.

  • 5
    Why is RAID "off the table"? – ewwhite Mar 18 at 14:25
  • storing the parity of equal sized disks on another disk as in RAID5 That's not really how RAID5 works. – Andrew Henle Mar 18 at 15:31
4

A checksum can only detect corruption. It cannot help you restore the data.

If you aren't willing to mirror your data on two (or more) vdevs, then you can set the copies ZFS property, which will tell ZFS to keep n copies of the data (1, 2 or 3). In case one copy is corrupted, the other can be used to recover the data.

$ zfs get copies srv
NAME  PROPERTY  VALUE   SOURCE
srv   copies    1       default

$ zfs set copies=2 srv

Keep in mind that this does not protect you from an actual disk failure (which can happen even if you have never experienced it before). This is not a backup. Make appropriate backups of your data.

  • 1
    Nor does having multiple copies stored in the filesystem protect you if you delete the file yourself. – Andrew Henle Mar 18 at 15:30
  • 1
    @AndrewHenle This is achievable via snapshots (and backups) – shodanshok Mar 18 at 15:39
  • "A checksum can only detect corruption. It cannot help you restore the data" is only true for a single simple checksum. More complex systems consisting of multiple checksums in the simplest case can be used to correct data. Look up Error Correcting Codes (ECC). Consider a table of bits, where every row and column is checksummed, Then when 1 bit is in error that row and column checksum will fail. The row / column intersection of failed checksums will tell you which bit to flip to correct the problem. It gets tricky to guard against multiple errors, particularly in the same row or column. – BeowulfNode42 Apr 10 at 5:21
  • 1
    @BeowulfNode42 This question and answer is specifically about ZFS. All of that is true, but completely irrelevant. – Michael Hampton Apr 10 at 14:45
  • 1
    ECC checksums are also extremely weak and prone to hash collisions that render them insensible to the fact that the data is corrupt at all. Modern hard drives almost univerally implement ECC checksumming at the hardware block level, which does not prevent them from cheerfully returning the occasional corrupt block anyway. – Jim Salter May 5 at 0:27
6

I don't think this is a good use case for ZFS. It's like you have a desired solution and are trying to make this fit.

Yes, there's the ZFS copies= parameter, which will endeavor to store multiple copies of files on different sectors of the disks, but it's not a substitute for true redundancy.

Please see: https://jrs-s.net/2016/05/02/zfs-copies-equals-n/

When you post a question and say such firm things like "RAID is off the table", it may be helpful to explain why you have those constraints so that we can help answer the real question.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.