3

I currently have two CentOS Boxes providing NFS Services to multiple Web servers.

Could someone recommand a file system which would mirror all files across both of these servers easily and efficiently?

I have used DRBD in the past however this had its drawbacks of only being mountable on one system at a time.

5 Answers 5

4

There is no way that I am aware of using free software to have a pair of two active NFS servers exporting an entire single clustered filesystem. If you are trying to achieve redundancy, you could set up a clustered filesystem and an active/passive pair of NFS servers following Red Hat's Configuration Example - NFS Over GFS (also see cluster administration). This gives you failover, but it's neither easy or efficient. If you were looking at a cluster as a way of increasing performance, this will be a step backwards because you've gone from two to one active NFS servers and you have the overhead of a clustered filesystem.

E.G. /gfs is a clustered filesystem mounted on both server1 and server2. server1 is used by all NFS clients that mount /gfs. If it fails, they'll be transparently switched to server2.

An alternate approach is to have a single clustered filesystem but for each server to only export part of it. Instead of a single active/passive pair, you have two NFS resources and each server is active for one of them. This won't increase your performance, but it will be less of a decrease in performance over your current setup since it lets have keep both NFS servers active.

E.G. /gfs is a clustered filesystem mounted on both server1 and server2. It is divided into two directories, al and mz. server1 is used by all NFS clients that mount /gfs/al. If it fails, they'll be transparently switched to server2. server2 is used by all NFS clients that mount /gfs/mz. If it fails, they'll be transparently switched to server1.

A tweak on the above would be using two DRBD resources instead of a clustered filesystem. This may be simpler to set up, especially if you're already familiar with DRBD.

7

I'd actually recommend Gluster. It is OpenSource, well documented and RedHat recently purchased it. It has a relatively good performance, and since it is a RedHat project now, it is well supported on CentOS. There also is a project called HekaFS, which has the goal of extending glusters Authentication and Security capabilites by adding SSL and other goodies. It is quite simple and very well designed. It comes with pretty good management utilities.

2

Checkout OCFS2: http://oss.oracle.com/projects/ocfs2/

Cheers

1
  • 1
    According to Oracle's page on OCFS2m (oss.oracle.com/projects/ocfs2), it is only available for Unbreakable Linux. I smell lock in and a desperate attempt to not support Red Hat or it's derivative, CentOS. I like it's list of features, just not its future. They seem to be doing the same thing with Lustre. That said, Red Hat is trying to lock people in with GFS, but I wasn't all that impressed under RHEL 5. Apr 5, 2011 at 4:04
2

Many folks have compiled the full list of options, both commercial and free, shared and non-shared disk. There are Wikipedia entries covering the high level clustered filesystem concept is here and an exhaustive list of filesystems.

It sounds like you are operating with non-shared storage by using disks local to each system. You will probably want a dedicated network connection between the two hosts so that synchronization traffic doesn't interfere with your normal web application.

Now my real problem is that you have two independent filesystems now and you presumably want to merge them into one, reliable, better performing filesystem, without causing any downtime to your web site. The tough problem is that you're about to cut your storage space in half and you'll have to go through some variation of these steps:

  • Select the filesystem of your choice
  • Build a test rig so that you can practice; replicate networking as much as possible
  • Test everything you see here on the practice system first so as to minimize downtime and mistakes
  • Migrate all the data to one NFS server
  • The one NFS server now serves both addresses
  • The second machine becomes the first node of a new clustered filesystem
  • All the data from NFS is copied to the new filesystem
  • Client migrations to the new cluster begin
  • Continuous migration of data from NFS to the clustered filesystem until all clients are on board
  • Drop the NFS filesystem
  • Rebuild the now-unused server and join it to the cluster
  • Ensure that replication and/or load balancing works as planned
  • Test the system by disconnecting the original node of the cluster and ensure that the clients stay up and running

If your only clients are a couple of web servers, then it might not be real bad. You can also continue to access the clustered filesystem over a variety of protocols like FTP, Samba, SSH/SCP, NFS.

One important tip - clustered filesystem make a poor storage location for database files of a live database. Use a clustering technology specific to your database engine if this applies to you.

The migration to a new filesystem is never short or easy. Good luck choosing your underlying filesystem and getting it in.

1

I've seen GFS2 (http://en.wikipedia.org/wiki/Global_File_System) running in production. You can pretty easily get up to 10 machines accessing it, and as long as you are not throwing obscene amounts of utilization at it expand beyond that.

3
  • And GFS is supportable under RedHat Enterprise Linux, so odds are that you can find other system administrators who know it if you ever decide to move on. Apr 5, 2011 at 4:01
  • Yup... and it plays nice in Centos and others. It's not a fantastic solution, however I have seen it handle large amounts of traffic with some high end tweaking.
    – tsykoduk
    Apr 5, 2011 at 5:45
  • And, when we used it, it was directly mounted by all of the target machines. You need a SAN/iSCSI for that, however those are not hard to come up with these days.
    – tsykoduk
    Apr 5, 2011 at 5:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .