Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Currently I cache responses using nginx + fastcgi. I'm in the process of moving my app to AWS. I'm architecting a horizontally scaled approach, but trying to determine where best to put nginx + fastcgi cache.

I found a few solutions from my own research, both with significant drawbacks.

  • Use AWS Elastic Load Balancer to balance a cluster of web apps. Each web app runs it's own nginx and web server stack. Drawback: the cache on each server needs to be warmed. Worst case 1/n the hit rate.
  • Put nginx + fastcgi in front of the AWS ELB so the cache is centralized for maximum hits. Drawback: single point of failure - if nginx dies traffic doesn't pass thru to the ELB. Added nginx instance.

I'd appreciate any suggestions of common architectures that overcome the drawbacks outlined above.

share|improve this question
You could always combine the two a bit, and have an ELB for a couple nginx+fcgi instances that pass requests along to the web server stack's ELB. – ceejayoz Jan 7 '13 at 22:24
@ceejayoz, true. That eliminates the single point of failure drawback, but combines the other two drawbacks. – Jason McCreary Jan 8 '13 at 1:40
Not necessarily. You can likely have fewer servers in the first tier, meaning fewer caches to be warmed up, and you've eliminated the SPOF that's the drawback of the first. – ceejayoz Jan 8 '13 at 5:36

I've set up this stack many times on AWS. Option #1 has always worked well for me and is the one I normally choose because it's the simplest. I'm curious to know how much traffic you're dealing with where the less-than-ideal initial cache hits are an issue? I'm serving a few million pageviews a month on a pair of m1.small instances and they're barely scratched.

Other thoughts:

  • Nginx can use memcached as a caching store. Plug it into an ElasticCache cluster. This will give you a very fast, centralized datastore for multiple instances to connect to for cache contents. (I've never done this, but it looks doable. Please post results if you do attempt it.)

  • The SPoF problem mentioned in Option #2 can be mitigated by setting up an autoscaling group that contains a single instance. If it dies, it should come back online in a matter of a couple of minutes. You will need to fiddle with a boot script that automatically grabs the Elastic IP though.

  • Keep in mind that putting anything in front of a ELB will require that you use VPC. This is generally good practice anyhow, but it can have a steep learning curve.

  • CloudWatch Alarm Actions were just announced today. You could do some cool stuff with this to help minimize the SPoF issue.

share|improve this answer
Thanks. I'll look into memcached as a shared caching layer for nGinx to mitigate the low hit ratio. The SPoF is a large concern given the role of this box (single entry). Minutes of downtime while another instance boots is unacceptable. – Jason McCreary Jan 9 '13 at 16:44
If mins of downtime isn't workable, you can run two identical instances, each in a separate AZ, and use a floating EIP. See this design pattern. – jamieb Jan 9 '13 at 17:13

Looks like you are thinking about the same architecture what I am planning to implement.

I have thought of both the scenarios and the same approach, and then settling up to this solution.

I will be using the 2nd option, setting up Caching solution in front of Load Balancer. I am going to handle the Single point of Failure problem with adding one more node having standby and using something like HAProxy to keep checking the health of the machine. If first Caching machine went down, then HAProxy will automatically move the IP and move the traffic to the other Caching Standby Machine and hence the Single point of failure will be handled.

Also, in my scenario, I will be using something like Varnish, instead of Nginx+php, which I will suggest you too, incase your application is not dependent on Nginx. Varnish will have it's own built in Load Balancer, hence you will skip the AWS Load Balancer too.

Hope this will help you building a robust and scalable infrastructure.

share|improve this answer
Thanks for sharing. Nginx has a built in load balancer as well. But using these instead of ELB does not offer auto-scaling, among other things. – Jason McCreary Jan 8 '13 at 18:43
Oh yes, that's right. My architecture is kind of fixed, so never thought about auto-scaling. Thanx for sharing that thought. – GeekRide Jan 9 '13 at 5:29

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.