Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I've got a web farm of IIS7 machines. works great. We're about to release an API to the wild. It's pretty simple but we know it's going to get hit pretty hard from day one (we have at least one signed up client).

So we're considering adding a caching layer BETWEEN our web servers and the interwebs. Firstly, I have no idea if this is a good solution, so I'm open to ideas. Secondly, what do we put in front of the farm? A dedicated Windows or Linux box? Our web load balancer is an F5 BIG IP.

I'm open to suggestions :)

Any ideas, folks?

share|improve this question
One thing to consider is to have the API service served off of a different set of servers from the primary web service. This way, if the API gets really hard, or vice versa, service for other functions is less likely to be interrupted. Otherwise, it's still a good idea to use some sort of reverse proxy to cache something that does not need to be computed every single time e.g. rss feeds. – Jauder Ho Oct 23 '09 at 11:54
up vote 3 down vote accepted

I'm not sure I understand the question. Reverse proxy caching is a common practice, and there are lots of tools for doing it. Varnish and Squid are popular in the open source world for doing this. Typically you'd put a couple of RPC boxes behind your load balancer, and then either go directly from the RPC boxes to your web servers or go back through the load balancer to the web servers.

That said, if you're talking about an API, typically most of the content you'd be serving would by dynamic, making the caching mostly useless.

share|improve this answer

I'm late to the party here; but I have something to contribute...

Cal Henderson talks (briefly) about the issues you're facing in his book Building Scalable Websites. You should read through the chapter on web service APIs.

As easel correctly points out, reverse proxy caching will typically not give much benefit for an API.

Two things that will benefit you are:

  1. A distributed key-value cache for the data that you're working with. I.e. you can cache your hot data in your platforms native object format, or as simple serialized arrays, or whatever else is fast for your platform. This limits the number of hits against you database, where you'll have your worst scaling problems. Your API servers will still need to fetch any non-cached datasets, and to serialize the response -- but at least the serialization part is only CPU bound, and can be scaled easily.
  2. Some sort of rate-limiting system, i.e. the capacity to throttle or deny clients who are making too many requests. API calls will typically involve fairly heavy processing on your servers (as they read or manipulate raw data), so protecting yourself against badly written clients makes sense.

In addition to the above, Cal Henderson suggests creating open source client libraries in popular languages, and putting best practices into these (i.e. client-side caching, rate limiting). This way 3rd party developers will have an easy, compliant code platform to re-use and build upon. The idea sounds great IMHO, but also somewhat costly.

share|improve this answer

A caching layer would help with static content but I do not see how it will help with dynamic APIs much. However, that can still be helpful.

For example, you could use a CDN to help cache and distribute static things to reduce the workload of your server so that it can spend more time on the dynamic stuff.

The for your dynamic stuff, you may want to employ something like what it mentioned here if you are using a CDN for dynamic stuff.

share|improve this answer
CDN's are good for large files. even though my api results (json) will be small and light .. their content will not change that often. Even if the content is 12 hours or 24 hours, it's still a lot better than hundreds of calls a minute for the exact same piece of json result data. – Pure.Krome Sep 10 '09 at 2:55
So, I guess you have your solution. Use a CDN? – sybreon Sep 10 '09 at 6:45
my api results are small and light so a cdn isn't that good IMO. – Pure.Krome Sep 10 '09 at 23:03
Is your problem a bandwidth/computation one? If it is bandwidth, it will help. – sybreon Sep 11 '09 at 0:47

CDNs like Akamai can be helpful for even dynamic content such as APIs and SSL. They do this by keeping open TCP sessions between their edge nodes and your servers. This reduces the number of open connections on your servers, and saves a ton of time and packets in establishing new TCP sessions.

A load balancer like HAProxy can also help by streamlining and optimizing TCP sessions. It can even run 1500 MTU packets on the 'World' side and Jumbo frames on the backend side to reduce processing times on your real servers. It has a number of tricks to offload connection handling from the real servers to the LB. Having threads and TCP sockets on your API servers tied up just for some slow TCP client is a waste of resources.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.