4

Our old 32TB RAID6 (net capacity: 26TB) system is showing its age and requires an upgrade, so we are currently thinking about building up a new 100TB system.

The storage is mostly used for our user's home directorie, for sharing project data (working with media, so projects can get up to 1TB) and for

The old storage is attached to a Debian host via iSCSI; the file server exports the data via NFS (for the un*x hosts) and SMB (for W32). For practical reasons, we would like to keep that.

In the past we found that project data grows at unexpected ends, so we use a flat storage pool where all the projects (and home dirs) are located in the same filesystem (rather than pre-allocating a certain amount of storage capacity to a given project, and then having to re-allocate when hitting limits). While this is a simple and flexible solution, I remember hitting some limitations even with the old setup. The biggest partition on the old RAID system is 20TB, because xfs wouldn't allow me to create a larger filesystem (back then; not to talk about ext4; and I didn't really consider zfs or btrfs - and still don't).

Since I haven't particularily followed recent progress on the storage front, I'm a bit lost what the current best-practices in creating such mid-sized storages are: JBOD, LVM, RAID, distributed filesystems; stacking which on which? Some blogs suggest that RAID6 is obsolete anyhow, but what are the alternatives?

My requirements are:

  • ~100TB net capacity
  • "flat" filesystem view
  • redundant (survives multiple concurrent disk failures)
  • white-box system
  • runs on Debian (Debian/stretch)
  • iSCSI

  • scalable: ideally such a system would allow for growing (e.g. I start off with 70TB and later buy disks on the go until I reach full capacity, and the stack scales along: the filesystem grows and data is still stored redundantly, hopefully without having a handful of separate RAID6 systems).

3 Answers 3

5

Go ZFS. If you want Debian ZoL is your best friend.

https://github.com/zfsonlinux/zfs/wiki/Debian

3

RAID 6 is far from obsolete - RAID 5 is however. I'm not a fan of single-controller storage, single point of failure you see, but if you're happy with that then by all means carry on with something similar to what you have now. Of course what you may well choose to go for now is some form of flash-based front-end - either through caching or tiering - it depends where you want to do the actual RAIDing - I like hardware controllers, you may not. Either way if you can get to a point where 90% of your IO is coming from/to flash (SSD/NVME etc.) then your users will be very happy indeed.

1
  • Well, RAID50 or RAID60 are still a good consideration. RAID5/6 are still good for all-flash storage arrays.
    – Net Runner
    May 10, 2017 at 11:37
0

Honestly it seems like you describe something similar to Linus's workload, maybe smaller file sizes but same linux based OS and large size requirements. Take a look at his storeanator review and give the guys at 45drives a look see.

1
  • Link-only answers aren't very helpful, links rot over time (many of my early answers met that fate). Please add at least a summary of the linked content so that this answer can stand alone.
    – sysadmin1138
    Nov 17, 2020 at 18:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .