Some background information:
In ZFS you build your storage pools out of vdevs (virtual devices). A single pool can have as many vdevs as you want, and each vdev itself can consist of one or more disks. Redundancy is managed at the vdev level, so your pool will always be striped (concatenated) over all vdevs it consists of. This means you will lose your pool if you lose a single vdev in it.
To prevent this, you normally do not use single disks as vdevs, but redundant sets: either mirrored disks (2, 3, or more disks which are completely mirrored/identical) or parity-based disk sets (RAIDZ1 with one parity disk equal to RAID5, RAIDZ2 with 2 parity disks equal to RAID6, or RAIDZ3 with 3 parity disks).
Some rules apply:
- You cannot convert or modify parity vdevs, except grow the disk sizes (not the amount of disks!) by replacing all disks with bigger disks.
- You can add and remove disks to or from mirrored vdevs and basic single disks (except the last disk of course) with
zpool attach
and zpool detach`.
- You can add new vdevs to any pool, but old data is not rebalanced automatically (only newly written data uses all vdevs). You cannot remove any vdev from a pool.
- You can mix and match different vdev types in a pool, although it is not recommended because your safety and performance is equal to the weakest/slowest vdev in the pool.
ZFS is not that flexible with parity RAID, therefore you should think about your data and growth and plan accordingly:
- If you want to be as flexible as possible, use mirrors because of attach/detach. Mirrors also have good performance and fast rebuild times. The simple solution is often the best one.
- If you want maximum storage space, you have to use RAIDZn: either populate all your enclosure slots with smaller disks and update them to bigger disks in the future, or buy the biggest disks now and use a subset of slots, for example RAIDZ2 with 6 or 8 disks and add another RAIDZ2 set of the same size later on.
For more in-depth information regarding the choice between RAID types and general pool layout, I suggest reading this blog post and also the ZFS best practices guide.