I am looking forward to use parallel file system using MPI in linux cluster. I am wondering if parallel file systems like lustre/Parallel Virtual File System require special hardware support(special hard-disks).
migrated from stackoverflow.com Jun 30 '11 at 1:11
This question came from our site for professional and enthusiast programmers.
For Lustre, the answer is "no" - no special hard disks required. Specific requirements (mostly software-related) are discussed here: http://wiki.lustre.org/index.php/Preparing_to_Install_Lustre
Some more details here: http://wiki.lustre.org/index.php/Configuring_the_Lustre_File_System
and of course, the FAQ: http://wiki.lustre.org/index.php/Lustre_FAQ
Rob Neely is correct in that there's no special hard disks needed, and he's selected good resources to get you started.
Let me just answer the implied other part of the question -- just because there's no special sauce needed in the harddrives doesn't mean that there are no hardware requirements. You can't just setup lustre on the same system you were using as a NFS file server and expect to get the benefits of a parallel file system like Lustre, PVFS, Ceph, etc. For a parallel filesystem to be a win, access needs to actually be parallel -- that means multiple systems for data stores (ODS, object data stores, in Lustre-speak; I/O servers in PVFS-speak; etc), metadata, and good network connections between the filesystem servers and the client nodes.
Still, those parts of the system are expandable, and you can start out small, find out what your needs are, and grow the system after you've got it working. But it's never going to be a software-only sort of deployment.