Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm setting up a HA (high availability) cluster using nginx, haproxy & apache.

I've been reading great things about nginx and haproxy. People tend to choose one or the other but I like both. Haproxy is more flexible for load balancing than nginx's simple round robin (even with the upstream-fair patch). But I'd like to keep nginx for redirecting non-https to https among other things right at the point of entry to the cluster.

On the other hand, nginx is a lot faster for serving static contents and would reduce the load on the powerful apache which loves to eat a lot of RAM!

Here is my planned setup:

Load balancer: nginx listens on port 80/443 and proxy_forwards to haproxy on 8080 on the same server to load balance between the multiple nodes.

Nodes: nginx on the node listens to requests coming from haproxy on 8080, if the content is static, serve it. But if it's a backend script (in my case PHP), proxy forward to apache2 on the same node server listenning on a different port number.

Technically this setup works but my concerns are whether having the requests going through several proxies is going to slow down requests? Most of the requests will be PHP requests as the backends are services (which means groing from nginx -> haproxy -> nginx -> apache).

Thoughts? Cheers

share|improve this question
add comment

migrated from stackoverflow.com Jan 5 '10 at 8:07

This question came from our site for professional and enthusiast programmers.

5 Answers

From a purely performance perspective, let benchmarking make these decisions for you rather than assuming -- using a tool like httperf is invaluable when making architecture changes.

From an architectural philosophy perspective, I'm a little curious why you have both nginx and apache on the application servers. Nginx blazes at static content and efficiently handles most backend frameworks/technologies (Rails, PHP via FastFCGI, etc), so I would drop the final Apache layer. Once again, this comes from a limited understanding of the technologies that you're using, so you may have a need for it that I'm not anticipating (but if that's the case, you could always drop nginx on the app servers and just use apache -- it's not THAT bad at static content when configured properly).

Currently, I use nginx -> haproxy on load balancing servers and nginx on the app servers with much success. As Willy Tarreau stated, nginx and haproxy are a very fast combination, so I wouldn't worry about the speed of having both on the front-end, but keep in mind that adding additional layers increases complexity as well as the number of points of failure.

share|improve this answer
add comment

Your setup is more and more common. You don't have to worry. Both nginx and haproxy are extremely fast to process and forward HTTP requests. Both combine very well together and do their job very well. No need to choose. Install them both and be happy. That way you will deliver static files very quickly and also ensure smooth scaling of your dynamic servers.

Don't worry for the number of proxies. The problem is often "can I use a proxy". Sometimes it's not practical. If you can have one, you can have two or three. Many complex architectures involve up to 5-6 levels of proxies and still scale very well. You should just be careful about one thing : do not install more of such proxies on a single machine than this machine has of CPU cores, or the proxies will have to share their CPU time under high loads, which will increase response times. But for this to happen with nginx and haproxy on a machine, this would mean loads of tens of thousands of requests per second, which is not everyone's problem of the day.

Also, avoid mixing single-threaded proxies with massively multi-threaded/multi-process software such as apache, java, etc on the same system.

Once you take these rules into account, simply draw the architecture that suits your needs, put names on the boxes, combine them in a sane way and install them.

share|improve this answer
add comment

Remember complexity can be just as much (if not more) of an impediment to scaling as code/design. As you scatter your implimentation details across more and more services and config-files you create something that is more difficult to scale out, has more of a learning curve to anyone new to the team, requires more software/packages to manage, complicates troubleshooting with more potential failure points, etc. Setting up a 4-proxy-layer stack for a site that would have been fine with just-apache or just-nginx is basically the sysadmin version of "premature optimization".

share|improve this answer
add comment

Why not use Varnish? That way you combine caching, proxying and loadbalancing into one application and it's a hell of a lot neater from an architecture point of view.
The scale-out scaling is pretty phenomenal. There's also the advantage that the load balancer can make more intelligent decisions based on the actual health of the nodes.

The configuration file will allow you to examine the headers and make decisions about where to serve static and dynamic content from.

If you're really predicting to be serving a LOT of static content, perhaps shunting most of that onto a CDN would be a cost effective solution?

share|improve this answer
    
Varnish doesn't do HTTPS, so you would still need something in front of it to terminate that. Nginx would be a good choice. stunnel or stud could also do it; i don't know how their performance compares to Nginx's, and they don't send an X-Forwarded-For header. –  Tom Anderson Aug 19 '11 at 11:34
    
I'd be very interested to read a comparison of HAProxy and Varnish for load-balancing. I'd like to be able to make an informed choice between {Nginx,stud}/Varnish/HAProxy and {Nginx,stud}/Varnish. –  Tom Anderson Aug 19 '11 at 11:35
    
@Tom Anderson, If you can wait, I could probably write one. Might take me a couple of weeks. –  Tom O'Connor Aug 22 '11 at 11:45
    
A kind offer! I should add that i am not facing this choice right now, but it's something i am conscious i don't have the information to make sensible choices about in the future. This seems to be a really tough area to benchmark. –  Tom Anderson Aug 22 '11 at 12:19
    
No comparison yet? It's been some weeks :) –  Henrik Jun 4 '12 at 17:51
add comment

"Technically this setup works but my concerns are whether having the requests going through several proxies is going to slow down requests?"

Yes, but not by much especially if you pay attention to the networking side of your scaled-out system. That has to be robust.

Generally, to be able to gain scale-out and spread work over many machines, you must usually sacrifice some performance over some or all of the individual machines. It's inevitable and usually something not worth worrying about (beyond the testing you should always do to ensure performance of the whole is sucking wind). Optimizing the parts is not necessarily the best way to optimize the whole.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.