Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I realize that since NFS is not block-level, LVM can't be used directly.

However: is there a way to combine multiple NFS exports (from, say, 3 servers) into one mount point on a different server?

Specifically, I'd like to be able to do this on RHEL 4 (or 5, and re-export the combined mount to my RHEL 4 server).

expansion
The reason I pegged lvm is that I want a bunch of exported mounts (servera:/mnt/export, serverb:/mnt/export, serverc:/mnt/export, etc) to all mount at /mnt/space so that my /mnt/space on this server (serverx) as one large filesystem.

Yes, I know that re-exporting is generally a Bad Thing™ but thought it might work, if there was a way to accomplish this on a newer release as opposed to an older one

From reading the unionfs docs, it appears that I can't use it over a remote connection - have I misread it? More accurately, since Union FS merges the contents of multiple branches, but makes them appear as one, it doesn't seem to go in reverse:
I'm trying to mount a bunch of NFS points in a merged fashion, then write to them - not caring where data goes, a la LVM .

share|improve this question

6 Answers 6

GlusterFS is very good for this job, you can also consider Lustre maybe (I have not used that one yet though) GlusterFS is NFS independent software but it would be very easy to move on that. You can also use it for Raid 10 networks which you might need in the future and it is very easy to scale.

share|improve this answer
    
thanks, Yuri - I'll check Lustre, too –  warren Nov 30 '09 at 8:14
    
this does look closest to what I'll need to implement –  warren Nov 30 '09 at 10:09
    
+1 for Glusterfs –  Tom O'Connor Apr 6 '10 at 19:27
up vote 3 down vote accepted

While remarkably hackish, the route I ultimately ended-up following is this:

  • Using VMware ESXi, add datastores that are NFS mounts (from whereever).
  • Create vdisks on those datastores
  • Add the vdisks to a VM running RHEL (because I'm used to RHEL)
  • Put all of the added volumes into an LVM
  • Export the LVM partition via NFS

Pros:

  • simple
  • cheap
  • easy to replicate
  • with dynamic disk extension via VMware, the space can all be "allocated", but not "used" yet

Cons:

  • requires yet another layer (the hypervisor)
  • if any of the NFS mounts drops, the LVM could become corrupted (an issue faced under any of the potential solutions)
share|improve this answer
5  
Reading about this kind of hack makes me feel diiirty. But not necessarily in a bad way. I'll have to keep this in mind someday when I try to do something just as crazy. –  Jed Daniels Apr 6 '10 at 19:27
    
@Jed Danies: like I said, it's sketchy as all getout... but it works - and well :) –  warren Apr 6 '10 at 19:37
3  
Why not simply create files on the NFS mounts and add them as loopback devices? That way you can skip the VMware bit which adds a lot of overhead. See mail-archive.com/debian-devel@lists.debian.org/msg220815.html for an explanation on how to do it. –  w00t May 7 '10 at 9:12
1  
BTW regarding the NFS mounts dropping, you could create a RAID5 set with dmraid, but you'd need to do some testing on how well that actually holds up. –  w00t May 7 '10 at 9:14
    
@w00t - software raid over nfs. interesting idea :) –  warren May 7 '10 at 12:13

Do you mean something like UnionFS (which isn't in any way analogous to LVM that I can think of), or just mounting several filesystems all next to each other (like /mnt/fs1, /mnt/fs2, /mnt/fs3)?

Also, re-exporting NFS mounts (and even NFS mounting filesystems that have filesystems mounted in them locally -- see the nohide option in exports(5)) is an exciting prospect, involving all sorts of corner cases and likely bug hideouts. "Here be monsters", indeed.

share|improve this answer
    
the reason I pegged lvm is that I want a bunch of exported mounts (servera:/mnt/export, serverb:/mnt/export, serverc:/mnt/export) to all mount at /mnt/space so that my /mnt/space on this server (serverx) as one large filesystem –  warren Nov 30 '09 at 3:16
    
and yes, I know that re-exporting is generally a Bad Thing.. but thought it might work, if there was a way to accomplish this on a newer release as opposed to an older one –  warren Nov 30 '09 at 3:17
    
if NFS is required, cannot think of anything other than unionfs (and family). –  sybreon Nov 30 '09 at 3:21
    
Yeah, UnionFS sounds like the ticket here. –  womble Nov 30 '09 at 3:27
    
from reading the unionfs docs, it appears that I can't use it over a remote connection - have I misread it? –  warren Nov 30 '09 at 3:53

I'm currently using IBM's GPFS on a HPC Linux cluster. Supports multiple direct attached nodes (we are using FibreChannel), others can have a network based block level access to the same volume.

share|improve this answer

You might also be interested in drbd+gfs.

share|improve this answer

How about iSCSI? A load of target machines, each presenting a block device to the initiator node? Then, on the iSCSI initiator, use LVM to joint the block devices together, mount this, and then export it as an NFS mountpoint?

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.