Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

One thing that has always perplexed me is storage best practices. Filesystems brag about how they can be petabytes or exabytes in size. Yet, I do not know many sysadmins who are willing to let a single volume grow over several terrabytes. I do know the primary reason behind this is how long it would take to rebuild the array should a drive fail. The more drives in a single LUN, the longer this takes and the greater your risk of losing another drive while the rebuild is taking place.

Then there's usage reasons. Admins will carve out a LUN based on how much space they think needs to be allocated to the project. It seems more practical to me for the LUN to be one large array and to use quotas. I understand this wouldn't satisfy every requirement (iSCSI), but I see a lot of NAS systems (NFS) managed this way. I also understand that the underlying volumes can be grown/shrunk as needed quite easily, but wouldn't it be less "risky" to use quotas rather than manipulating volumes and bringing possible data loss into the equation?

There may be some other reasons I'm missing, so please enlighten me. Can we not expect filesystems to ever be so large? Are we waiting for the hardware to get faster to cut down on rebuild times?

share|improve this question
up vote 1 down vote accepted

Spindle separation is a good reason not to have one large volume. If you are running Exchange, SQL Server, and other random things (ESXi guests maybe) off of a single NFS device you will certainly want spindle separation. Exchange data and longs should be on separate spindles from eachother as well as SQL databases and logs.

share|improve this answer

To rebuild a drive doesn't depend on the file system but on the drive itself. If you used 2TB drives, then you'll need a lot of time to rebuild.
Rebuilding is a problem of raid 5. For this now may controllers support raid 6 or much better you could make a raid 10 (if you want the best performance). Exporting a lun is a job for a SAN, exporting a file system is a job of NAS.
Both have their pros and cons (search on google, there're tons of paper about). Making a lot of independent luns (or file systems) has the advantage of snapshot backups.

share|improve this answer

Personally I wouldn't run true enterprise traffic (Oracle, MSSQL, Production ESX) over iSCSI or NFS at all - I know and trust FC, both for general performance and during failure situations. Oh course this means that I can't just create massive LUNs and subdivide and it does create quite a bit of complexity/work but our system availability and end-user-experience-figures have been better and more consistent than we'd hoped.

As for your actual answer - well your posts seems very NetApp/filer-oriented, hence my last paragraph - but you've laid out the reasons yourself. Disk interfaces have fallen significantly behind disk capacity - meaning that rebuild times can be ridiculous, especially if you want to see decent performance during a rebuild to your existing traffic. Disks die all the time and the idea of affecting multiple platforms performance during a rebuild to simply make life easier for the admin would be a short-lived method indeed. Also purely from a platform security perspective you might want to partition your systems and the 'big LUN' approach may well not work in that scenario (based on hardware of course).

Ultimately Enterprise SANs are inherently business or mission critical, demanding availability and consistency over cost, ease of admin or even ultimate performance.

share|improve this answer
About DBs and NFS, you can see a lot of people are using oracle on some NFS exports. Actually Oracle is pushing this setup – Pier Jun 7 '10 at 14:07
And it seems like netapp is pushing iscsi quite heavily lately also! – tony roth Jun 7 '10 at 14:40
Their both a good compromise, and many people are thoroughly behind them both for their value - but aren't often used by large organisations for their business/mission critical systems. – Chopper3 Jun 7 '10 at 14:46

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.