What range is considered safe for connections/GETS/SETS per second per server instance with memcached?

Question

Question

I am interested in learning the following metrics, per memcached instance:

• Safe range for connections/GETs/SETs per second
• Upper bound for connections/GETs/SETs per second

I have a feeling the true bottleneck will be connections, but would like input from people who have established installations of memcached on their sites.

Background

I run a website that is seeing hundreds of millions of page views over the course of a month. It is distributed across multiple web servers. The site was originally coded with a file-based cached schema that was not shared by the web server pool; each web server maintained its own cached copy of each page.

For obvious reasons, we're migrating to memcached. We converted our lower-traffic but more-dynamic pages (aka 'pages with lower cache hit rates') without issue. We're now moving on to our higher-traffic but more-static pages (aka 'pages that should see higher cache hit rates'). We've converted the lowest-traffic ones first and we've already seen a jump from 3.5k GETs per second on average to 11k GETs per second on average. We're seeing between 400-600 connections active at any given time on average. Our connection limit is set to 4k in our configuration file.

Considering we still have the highest-traffic pages left to implement, this seemed like a good time to research the accepted ranges and upper limits regarding memcached. That way, we can determine if we need to expand to additional memcached instances before we move the highest-traffic portions of our site over to memcached. I realize our usage now is no cause for alarm, but I'd like to know when it will be, and I'd like to know that beforehand.

Answer 1

A number of concurrent connections is more important than a number of GETs/SETs from my experience. I am looking at a historical Cacti graph right now, the graph states that memcached instance got about 4M accesses per second at max (2.8M GETs and 1.2M SETs). I doubt these numbers are real. They were achieved using only one active connection anyway. The problem was that when I stress tested this setup using a web site load testing tool, memcached started to hog CPU at only about 2-3K accesses per second. I had to build a farm of memcacheds to distribute the load. As you can see, there is a non-linear relationship between number of concurrent connections and number of accesses per second memcached can handle. Memcached seems to start degrading quickly on a certain number of concurrent connections, you really need to stress test your setup to determine this critical number.

Answer 2

Perhaps a better way to look at it is to see your limitation on performance interacting with memcached. Some scripts to ascertain the average response time across your typical day would be a good place to start.

On your main question:

I think in general the amount of concurrent activities (gets/sets) will vary on your hardware configuration anyway so any responses on that would be subjective. I think finding a better way to measure the success relative to your own environment to get right to the point. As noted, stress testing is naturally very important in that process.

Cheers

Answer 3

There's no magic formula for predicting the capacity - you need to do this by experimentation and numeric analysis.

we've already seen a jump from 3.5k GETs per second on average to 11k GETs per second on average

If you're saying that this is the result of implementing the change, that implies that you were previously losing 2 thirds of your traffic due to performance issues?

While these metrics suggest you may be able to service more traffic with less hardware, did you measure the effect on response times? Using appropriate network simulation?

You need to plan an architecture which allows you to control what and where the caching is done - you want to be able to easily distribute the load across multiple memcached instances with some capability for failover. This gets very complicated, very quickly - which is one of the reasons I tend to leave memcached till a last resort - and never for a front-end cache. The other reasons is that where I've tested it, there is negligible gains compared with other methods of sharing data (reverse proxies at the front, good tuning in the middle, and distributed, shared storage at the back end). But I'm very conscious that the architecture needs to reflect the components used to build the system - my experience is mostly with mid-scale LAMP type systems - but model with a split application server tier is very different.