Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a Redis server running version 2.4.5 and with a dump.rdb of 11GB loaded into memory. It is running on EC2 on a high memory 4x extra large instance (70GB total memory). However, turns out Redis is already taking up 50GB of memory and is just growing more and more. My dataset is still gonna grow larger, probably to around 20GB, so clearly 70GB memory wont be enough. Do you guys have any ideas on how to overcome this limitation or how to make Redis eat less memory?

I've tried redis 32bit but it dies trying to load the data set into memory at startup.

Have also tried max-memory in the past but got weird results. Haven't tried virtual memory since I read it is/was gonna be deprecated.

share|improve this question
    
Buy More Servers? Buy more memory? –  Tom O'Connor Dec 30 '11 at 9:23
add comment

2 Answers

up vote 3 down vote accepted

Buy another server, do sharding. This is how Redis is intended to work. If that isn't that what you want, then Redis is the wrong software for your purpose.

share|improve this answer
add comment

If you have adequate swap, you can just let it run out of memory. Performance will suffer. If you don't want to keep your entire dataset in RAM, redis is the wrong tool for the job.

See the Redis FAQ:

You may try to load a dataset larger than your memory in Redis and see what happens, basically if you are using a modern Operating System, and you have a lot of data in the DB that is rarely accessed, the OS's virtual memory implementation will try to swap rarely used pages of memory on the disk, to only recall this pages when they are needed. If you have many large values rarely used this will work. If your DB is big because you have tons of little values accessed at random without a specific pattern this will not work (at low level a page is usually 4096 bytes, and you can have different keys/values stored at a single page. The OS can't swap this page on disk if there are even few keys used frequently).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.