This has been asked a million times but every time the answer is subject to "it depends on your requirements" condition, so I cannot extract a general guideline to apply to my case. So I ask again.
I have a 24 bays disk server (dual Xeon Silver 4210R with 128 GB RAM and CentOS 7) and 16 TB disks to store scientific data, organized in large files (size ~ GB) which are typically written once and then processed many times (the output of this processing does not matter for what follows). Data is mission critical but to some extent recoverable from other storage sites, so a failure with data loss is a big issue but probably not a killer. Available disk space should be maxed within the previous constraints. To summarize, in order of decreasing importance my constraints are:
- data integrity
- read performance
- available disk space
My tentative solution is to use a hardware RAID 60 level with two RAID 6 arrays of 12 disks each, and the ZFS filesystem. In my poor understanding, RAID 60 should provide a more reliable and read-performant solution than RAID 6 with a reasonable loss in terms of available space, and ZFS is a good choice for a fault-tolerant filesystem. I have no clue about possible downsides of this configuration (e.g. array rebuild time? A different filesystem?) nor about possible better alternatives, so I'd like to hear some informed opinion.
Thanks in advance for any suggestion.