Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a situation where a large batch of servers (X), on demand, need to request data from a smaller set of web servers (Y).

The worst case scenario is if all servers in X decide to fetch different requests to one server in Y. That would be X amount of connections, which could be a very large burst of traffic. The best case scenario is if 1 server in X hit 1 server in Y in tandem. Life does not work like this.

One idea to entertain is placing a proxy, similar to squid between X and Y. All of X servers can connect to this proxy, but would result in a few persistent (http keepalive) connections to Y. If The few were say, 3 or 4, then it would funnel. If we could then rate limit those connections and traffic decides to spike unusually high, we wouldn't hurt anyone but ourselves.


share|improve this question

Have you considered load balancing? There's a variety of technologies that accomplish this. My preference is LVS but others solutions applicable to Web server load balancing include..

LVS has different schedulers, which specify exactly how connections are handled. It's also very lightweight, I run it on small commodity hardware handling the traffic for a high volume Web site.

share|improve this answer
Thanks for the quick response. I'm still digging in, but I'm not finding an umbrella configuration of say, "outbound connections per ip/host" in the proxy sections of NGinx or Pound, but I'm still looking. It may be too specific of a requirement. We'll see. LVS feels a bit too low level. I'd need something that works on layer 4/http layer. – spencer p Jun 14 '10 at 20:45
LVS can work on the application layer but it depends how far you want to go and how you want to manipulate it, as there are many tools that may be better suited. For connection throttling, I would do that on the network layer. Netfilter (iptables) is capable of doing this. – Warner Jun 15 '10 at 2:42
Take the model a registrar works. I perform a search on "foo", it goes out to all the registry's api's and asks, "is available" "" etc etc.. Because each registry is one company, it'd be nice to pool by registry for instance without having to add a new registry by IP into a config file. This whole line of thought is in that vein, http b2b webservice calls. – spencer p Jun 15 '10 at 13:59

Have a look at nginx in a reverse proxy/caching configuration or if you're more comfortable with Apache, Apache2 + mod_proxy, mod_proxy_balancer and mod_cache.

share|improve this answer
ProxyPass almost gets it right for me. I'd rather not have to configure ea url but do something more global like, "Based on any class c, or hostname, or some other metric, limit the number of connections to 5". If that was there, I could weed out edge cases for specific hosts/ips with the existing ProxyPass hopefully. – spencer p Jun 14 '10 at 20:49

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.