Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We're considering hosting a web application on Amazon AWS. I've drawn a proposed setup for it, which I will try to summarize in a few lines:

  • The web application is served from webservers in 3 Availability Zones behind a loadbalancer. The webservers are automatically scaled up from 3 when the load increases. This is done through a user data file that runs a few bash commands.
  • The database is placed on a Multi-AZ RDS solution
  • Because the application writes to the file system, we also need to mount some kind of network attached file system on the webroot.

The last bit is what worries me. I have some experience with AWS and besides dealing with the latency between two Availability Zones, this would provide a single point of failure.

So, I've been looking at GlusterFS, because that's what someone on serverfault suggested to someone who was dealing with a similar pickle. I've been setting up an environment with a Gluster node in every AZ. In the startup script for my webservers, I evaluate the name of the AZ it's in and pick the Gluster node that's in the same AZ, to reduce latency. That's perfect!

But let's say the node in AZ us-east-1a fails somehow. Is there a way to get my webservers in us-east-1a to fall back to the node in us-east-1b if the node in us-east-1a is unavailable? And, of course, if both are unavailable, also to us-east-1c?

So far, I've only seen examples of people using the server and client functionality of Gluster on the same machines, which I'd like to avoid. It's probably good to note that I will be using the NFS client for performance reasons.

Of course, any other suggestion for this file storage system would be very welcome.

share|improve this question
1  
does that have to written to a filesystem? it could be easier to write to S3 –  Javier Aug 13 '13 at 20:04
    
You should see if S3 will suit. If every millisecond of performance counts GlusterFS may be a better option but you should consider (and rule out) S3 first as it is easier to work with and already scales etc. –  Drew Khoury Aug 14 '13 at 0:23
    
I'm not fond of S3. S3 doesn't support custom SSL certificates (which could be overcome if Cloudfront custom SSL certificates wouldn't set me back 600 bucks a month (4 times in our case)) and it's hard to migrate away from. –  Jaap Haagmans Aug 14 '13 at 6:52
add comment

1 Answer

up vote 1 down vote accepted

I'd like to update anyone interested on how I decided to go about this. Like I said, S3 doesn't suit our situation which is probably something more people struggle with.

Gluster seemed the way to go at the time, because it's specifically meant to do these things. However, in our testing environment, Glusters speed held us back. Yes, file transfers to a Gluster filesystem are pretty fast, but we do a lot of lookups and quick reads and writes on these mounted volumes and when testing for higher load, it became a bottleneck. The Gluster NFS client is much faster for these operations, but it doesn't support Glusters built-in fault tolerance.

So, we went back to our original idea: Simply using an NFS server with a failover in another AZ. To keep the nodes in sync, we use DRBD. We got around the problem of not having a virtual IP using VTun (I'm very open to other suggestions) and are using Heartbeat to promote the slave when the master is down.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.