Im a part time systems administrator and there is little formal documentation. Our main file server was set up in 2010 by somebody who is not even at the institute any more. The requirement at the time was to have three shared filesystems:
/home with quota per user and backup,
/usr/remote for large software packages and a
/scratch without a quota and no backup. Digging through the disks I found an LVM and a RAID controller with a proprietary software.
The server apparently has 14 disks with 1 TB capacity each. They are linked up in pairs using RAID 1. The RAID controller (Areca Technology Corp. ARC-1280/1280ML 24-Port PCI-Express to SATA II RAID Controller) is capable of RAID 5 and 6 (specs), but somehow that has not been used. Then one pair is used as the root partition for that server,
/. The other six pairs are linked into a single LVM group and that group then has the three partitions.
I asked about the reasoning and one of the senior people could only tell me that the person who set it up supposedly knew a fair share of LVM and had really thought about this. But I fail to see the advantages of this setup. I only see disadvantages:
We have 12 disks with 1 TB of raw storage, with the pairing up we end up with only 6 TB of actually usable space. Using RAID 5 with all of these disks we would have gotten 11 TB of storage, with RAID 6 it would have been 10 TB. Even if we took two sets of six disks into RAID 6, then it would still be 8 TB of usuable space.
The worst case disk failure is a whole pair, then part of the LVM is lost and I would think that basically the system would be screwed then. The best case disk failure is one of each pair, so that would be seven broken disks. So from a redundancy standpoint we are not better than RAID 5, given the worst case.
With RAID 6 we would have two redundancies and still could use much more space compared to the current setup. So what is the crucial advantage of this that led people to set it up in this way?