0

I have a load balancer/reverse proxy (either nginx or cherokee, does not matter which) deployed on my server and it points to multiple background webserver processes (either gunicorn or cherrypy, does not matter which) in a round-robin set-up.

In order to minimize downtime, I have a webserver restart script that kills a specific webserver process (out of say 8 processes) and then starts it up immediately again; and then goes on to the next webserver process (kill it, then start it up again) so at any moment in time, there will always be at least 7 processes available for my reverse proxy to point to.

This is great; but is there a way to "perfect" this process such that I do not get 502 Bad gateway at all? 502 gateway happens when a user so happens to be on the site and using a webserver process that gets temporarily killed and revived.

Obviously, the reason why I need the restart script is for deployment of new python code in my python application (running on either gunicorn or cherrypy).

4 Answers 4

3

nginx should do this out of the box. proxy_next_upstream's default error timeout setting will pass a request on to the next member of an upstream block if the initial server chosen is unreachable.

3

Why are you killing gunicorn? Just send it a SIGHUP like any other well-behaved Unix process and it will happily reload without losing any connections.

1

If you use haproxy and configure a backend for each webserver process and also use health checks on those backends, it will detect when one is down and mark it as offline, balancing requests to the healthy ones instead.
A user connected to the restarting webserver should be rebalanced to a working one after he refreshes the page or initiates another request.

0

Use apache httpd instead.

It has an ultra-configurable reverse-proxying loadbalancer that supports member drain and weighted preferences, not to mention online reconfiguration through the balancer-manager interface.

4
  • That won't solve my problem whichever loadbalancer I use. The problem (if it even can be solved) is to somehow kill and revive my background webserver process (the processes behind the loadbalancer) so quickly that the user won't even find out. Jul 25, 2012 at 8:47
  • You missed the point, then.
    – adaptr
    Jul 25, 2012 at 8:49
  • O... you mean that with apache httpd, it is possible to make a BalancerMember offline first, restart the background process and then put the BalancerMember online again? And while the BalancerMember is offline, the user is routed to another background process? Jul 25, 2012 at 8:51
  • 2
    ...that is how it works, yes :)
    – adaptr
    Jul 25, 2012 at 9:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .