Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a pool of 28 2TB-disks (56T) in 4 arrays of 7 disks. Since it's raidz1 (~RAID5), I'd expect 1 disk to be used for parity in each array, so the resulting volume should be 2TB*4*(7-1)=48TB, right?

Now, what I see on my system now:

$ zpool list volume
NAME     SIZE  ALLOC   FREE  CAP  DEDUP  HEALTH  ALTROOT
volume  50.5T   308K  50.5T   0%  1.00x  ONLINE  -

$ zfs list volume
NAME     USED  AVAIL  REFER  MOUNTPOINT
volume  2.00T  40.3T  75.8K  /volume

$ df -h /volume
Filesystem             Size   Used  Available Capacity  Mounted on
volume                  42T    75K        40T     1%    /volume

So, there are only 42T instead of 48T. Where are the missing 6TB? :) And where does the number 50.5T come from?

Please help me understand!

share|improve this question

3 Answers 3

up vote 4 down vote accepted

A 2TB disk is not 2 TiBi in size - it's only 2*10^12 / 2^30 ~ 1862 GiBi.

4 arrays of 6 effective disks each would be 24 * 1862 = 44703 GiBI, or 43.6 TiBi of real, usable storage.

I reckon it has some additional overhead you're not taking into account - IIRC RAIDZ also does snapshots and scrubbing, which take up additional space.

share|improve this answer
    
+1 Rounding, snapshots, reservations, other file systems, etc can all play into the slightly "off" number. Those are functions of ZFS, RAIDZ is a part of zpools, which are part of ZFS. –  Chris S Jan 20 '12 at 18:03
    
Note: The binary prefix for "tera-" is "Tebi", not "Tibi". And "Gibi" should not have a capitalized "B". –  Chris S Jan 20 '12 at 18:04
    
That sounds perfectly reasonable! Thank you very much for this explanation, guys! –  Pavel Jan 20 '12 at 18:06

To clarify the discrepancy in output between the commands:

The zpool command counts the disks that are being used for redundancy as space, while the zfs command does not; thus, the 50.5 TB number is your raw disk size, while the 42T is after taking out the 4 disks for redundancy.

share|improve this answer
    
Thanks, that is exactly what I needed to know! –  Pavel Jan 20 '12 at 18:09
    
...except the after-RAID figure is 43.6, not 42. –  adaptr Jan 20 '12 at 18:10
    
@adaptr Indeed; as you mentioned, something is fuzzy with its math (though no snapshots are listed so it's not that, and scrubbing is not a space spend - but there's definitely some extra usage somewhere). I saw your answer come up before mine, and only added mine to help clarify the difference in output between the zpool and zfs commands. –  Shane Madden Jan 20 '12 at 18:17

Hard drive manufactures measure disk size in Base 10. Computers measure bytes in Base 2.
It should say 42 TiB to clarify the SI use of the tera- prefix.

share|improve this answer
    
thank you! Of course you are completely right. –  Pavel Jan 20 '12 at 18:07

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.