8

The general rule of thumb is don't use more than 80-85% of your capacity, otherwise performance degrades sharply, and other bad stuff may happen.

On a 8TB array, this comes out to around 1.6TB of space that must be left free! A 40TB array would require about 8TB of free space!

Is this still the convention with larger arrays? Am I leaving way too much disk space on the table?

Improve this question
1
  • 2
    An interesting trick I saw on the ZFS On Linux discussion list is to create a zvol or fs with a reservation amounting to the disk space you want to leave free on the pool, and then not use it for anything. With a file system, you could perhaps set mountpoint=none canmount=no reservation=N. As counterintuitive as it sounds, it should work: as far as any other part of that pool is concerned, that space is not available (because it is reserved), but the block allocator is allowed to work with it (because sans snapshots new blocks written cause the old blocks to be deallocated; net result zero).
    – user
    Jul 16, 2014 at 14:05

2 Answers 2

Reset to default
6

Yes, it's still the convention, and yes, it holds true even as you scale. With ZFS, in fact, you really don't want to get to the 75% mark in your zpool too often. Fragmentation, snapshots and general performance tend to be impacted.

If building anew, don't start with anything more than 40% utilization and be sure to plan for growth.

Improve this answer
8
  • Yes, it's a fresh build with FreeNAS running on top with ZFS RAIDZ2. -- I just wanted to make sure I wasn't going to sound like a crazy when I tell management we need to buy a few extra hdd's just to have them sit empty all the time ;-P
    – SnakeDoc
    Jul 15, 2014 at 18:46
  • 4
    When you fill your fridge/freezer with food, do you leave a bit of space empty to allow you to rearrange things to get at the back? Or pack everything in 100%?
    – MikeyB
    Jul 15, 2014 at 18:50
  • 5
    @MikeyB Oh, I always pack it in there ;-) How else can I fit all that beer?
    – SnakeDoc
    Jul 15, 2014 at 18:51
  • Is this true for any OS / Raid Array combination?
    – mxmissile
    Jul 15, 2014 at 22:24
  • 1
    For ZFS or any other copy-on-write filesystem, yes.
    – ewwhite
    Jul 15, 2014 at 22:25
3

It really depends on your usage, though. Big files that are written once, and then only read? Doesn't fragment, can fill up closer to capacity.

Traditional UNIX home directories with lots of small files, some rewriting, lots of snapshots. Maybe even snapshot retention that keeps lots of recent snapshots, but removes some of them while keeping some older ones. Terrible fragmentation, pool really needs free space to perform.

The only way to know for sure is to test and monitor the performance.

Improve this answer
1
  • This box will be hosting some virtual disk drives for the Xen cluster over iSCSI, but will also likely have a CIFS share on it for office file sharing. So, there will be a mix of large files that have constant moderate I/O and some mostly static files that get read more then written to.
    – SnakeDoc
    Jul 15, 2014 at 19:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .