Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm planning to use distributed Memcache across a series of Linux web servers on Amazon EC2. These EC2s currently run auto scaling so will increase and decrease with load.

I have used this post to guide the initial setup.

best practice with memcache/php - multi memcache nodes

And another link it references in a reply.

http://techgurulive.com/2009/07/22/a-brief-to-memcached-hash-types/

I'm a little unsure on how the web servers local AWS IPs will be updated in the list of nodes within the Memcache client code as and when the servers increase and decrease.

I am currently using PHP PECL - Memcache on the client.

Does anyone know the best implementation to manage this with the AWS environment?

Thanks in advance for any suggestions.

Dave

share|improve this question

2 Answers 2

It sounds like you are designing a fairly large system to handle quite a bit of load. If so, you likely want to have separate instances for Memcache and web nodes, as their performance characteristics are quite different. By separating the two you can use an instance type more suited to each.

Memcache uses almost no CPU under normal circumstances, so using a high memory instance makes sense. Likewise, web servers are generally processor intensive. Use high CPU instances for them.

Keep in mind that memcached itself is very simple. It sits and waits for a command saying "get me this key" or "set this key to this value" (there are a few others, but that's the basics). Any hashing/etc is purely done in the client memcached library.

With this in mind, auto-scaling of memcached is quite ambitious, although it certainly can be done. If you are constantly bringing up and decommissioning servers, you will either have to accept that some cached data will periodically be lost, presumably putting more load on your databases, or ensure that all data stored in a memcached instance is first copied to another instance.

share|improve this answer

A bit late on this but you don't really want to have memcache nodes autoscaling. Hashing will drift so you will be losing large percentage of your cache everytime there is a scale up or scale down event.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.