Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Any hard numbers, or rules-of-thumgs, for what workload (e.g. in terms of simultaneously active users running average HPC center jobs) on a computer cluster, that would make a parallel network shared file system such as Lustre, GPFS, Ibrix or Panasas, a requirement?

...that is, when is an NFS shared SAN storage not enough anymore, and you need some kind of parallel NAS?

EDIT: Added "NFS shared" before "SAN".

share|improve this question
What do you define as an "average HPC job"? What is it doing? How much network and file activity does it generate? I'm note sure there is such a thing, and even if there was I'm not sure it would allow us to answer your question. – Sam Sep 13 '11 at 14:18
Well, at the HPC center I'm thinking of (UPPMAX) we have a very varied assortment of users; Big bunch of physicists, some quantum chemists, a few staticians, and LOTS of biologists, running everything from sequence assembly to blast searches ... so I was thinking that this assortment would form some kind of "average", because of it's wide assortment. Understand that it is hard to form an absolute general average, but at least we should not be too heavily leaning towards any one kind of "user dynamics". – Samuel Lampa Sep 13 '11 at 15:07

First off, there is no "average" in HPC that I'm aware of - I've had the pleasure of working in three HPC environments, and all 3 had wildly different dynamics (in classical bottleneck-modeling terms one was disk bound, one was CPU bound, and one was memory-constrained).
This leads directly to my answer to your question: You need "some kind of parallel NAS" when you are disk bound to the point that it impacts performance. You will know this is the case when you can no longer keep all of your CPUs saturated (and you still have free RAM, or some nodes are completely idle waiting on the disk).

If your current storage system has performance monitoring features you can watch your statistics and plan to take action when you get to say 66% or 75% of the maximum performance values, but there is no hard and fast rule that says "X nodes, Y tasks - now you need to deploy GPFS."

share|improve this answer
Many thanks for your advice! I answered some on the HPC "average" in a comment to the main post above. – Samuel Lampa Sep 13 '11 at 15:09

You are comparing apples and oranges. A SAN provides logical volumes - not filesystems. If you've got more than one system trying to mount the same logical volume as writeable (i.e. even with access time enabled) then you need a filesystem which supports concurrent access.

Typically the term NAS is applied to a system which provides file sharing services such as NFS or SMB which makes using a cluster filesystem redundant (only the controller accesses the filesystem directly). However the term is sometimes applied to the iSCSI protocol - in which case the device is behaving like a SAN.

share|improve this answer
True ... I realize I was thinking of an NFS shared SAN. Have updated the post. – Samuel Lampa Sep 14 '11 at 11:14

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.