Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

The setup looks something like this.

iSCSI setup

The goal is to have multiple computers mount a single LUN from the iSCSI target. Ideally mounting read/write and with ACL support.

The servers are running GNU/Linux, so preferably a filesystem available in the vanilla kernel. Though I can deal with compiling third-party modules if necessary.

I'm currently looking into GFS2 and OCFS2. What are some successful deployments people have done like this? Any gotcha's I should look out for?

share|improve this question

So long as the iSCSI stack you use has very strong SCSI3 reservation support, you should be good. The implementation details of the filesystem shouldn't matter. The biggest concern is generally the quality of the iSCSI target software, since bugs in that will lead to faults in the filesystem and overall faulting of the clustering. You want something that fails cleanly in those cases, and all of the generally acceptable clustered filesystems do that.

share|improve this answer

I have successfully tested GFS on Red Hat Cluster Suite in LAB. The fencing device use SNMP IFMIB to shutdown network port, you can also add quorum disk for extra security. You can find some tech notes here

GFS depends on Red Hat Cluster Suite, which is not easy to implement. If you don't need really cluster feature, just need to write to a shared block device concurrently, check IBM GPFS, it much easier to implement and the license fee is only few hundreds bucks.

share|improve this answer

I have 8 web / app servers mounting multiple OCFS2 filesystems. The common stuff, Apache configs, logs, vhosts, etc and app code are stored there. I wasn't around for the initial implementation, so I can't speak much to that, but from what I remember off the top of my head, it's pretty standard with some modifications.

As for gotchas, it can be somewhat fragile. By this I mean, we had many instances where when a server left and rejoined the cluster, the load on all the servers would shoot through the roof for a few minutes. I think this was a bug though, IIRC after some updates the behavior is resolved.

The cluster configuration file must be maintained in the same state on each node, which can be a pain.

For us, updates are going to be a problem. We are on 1.4 I think, and an update to 1.6 means downtime for the entire cluster since the versions aren't compatible. You can't mix the two versions. Maybe on a anew deployment this isn't such a big deal.

Also, keep in mind a cluster fs is going to incur more overhead than a conventional fs. The more nodes you add, the larger the problem. The solution is more hardware (and good tuning).

share|improve this answer
Have you had problems moving from 1.4 to 1.6? Sometimes reality does not match the docs. The OCFS2 project page says, "OCFS2 1.6 is fully compatible with OCFS2 1.4. Users can upgrade the cluster to the new release one-node-at-a-time (rolling upgrade)." – Pablo Maurin Feb 4 '12 at 16:37

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.