Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm looking for a data store that is very memory efficient while still allowing many object changes per second and disregarding ACID compliance for the last X records.

I need this database for a server with not much memory and I can make a key-value store, document, or SQL database work.

The idea is that indexes/keys are the only thing I need in memory and all the actual values/objects/rows can be saved on disk do to the low read rate (I just want index/key lookup to be fast). I also don't want records constantly being flushed to disk, so I would like the last X number of records to be held in memory so that 100 or so of them can all be written at once. I don't care if I lose the last 10 seconds worth of objects/values. I do care if the database as a whole is in danger of becoming corrupt.

Is there a data-store like this?

share|improve this question
    
This sounds like memcached, but I'm not that familiar with it so I'll let someone else confirm –  Mark Henderson Jul 2 '11 at 22:44
add comment

3 Answers

Redis can handle this situation -- it writes everything to disk (eventually), but you can specify how frequently it does this, and anything not written is (naturally) lost.

http://redis.io/topics/persistence gives you all the gory details.

share|improve this answer
    
The problem is that Redis requires the data to be stored in RAM also (or swap space). You cannot have a Redis database with more memory than RAM + Swap. It's the same problem that memcached has. I can't fit 2GB of data in a membase or redis database on a 256MB VPS. –  Xeoncross Jul 8 '11 at 17:09
    
Hmm, I misinterpreted your question then. –  womble Jul 8 '11 at 22:53
    
Redis comes the closest. If only they didn't keep the data in memory. –  Xeoncross Jul 9 '11 at 1:38
add comment

Yes - any database using asynchronous I/O on a modern operating system will have near optimal caching.

share|improve this answer
add comment

Redis has already been mentioned

Other options:

  • scalaris (somewhere on code.google.com) -- the complete CAP and is distributed
  • riak (basho.com -- eventually consistent, distributed)
share|improve this answer
    
Scalaris looks nice, but it doesn't store data on disk so you would need a node network setup if you didn't want to lose your data. –  Xeoncross Jul 8 '11 at 17:26
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.