Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Briefly: if I have 5 Tb of data and want to deploy this on 5 cassandra servers - does each machine need to have 5 Tb of disk space for data (not counting log space)? From the docs it sounds like at times cassandra will need 2x the data size - so 10Tb / server or 10Tb total in the array?

How much RAM should each machine have? Assume that the 5Tb is all in the same column space. I had been planning to max out the RAM on each machine but I'm not sure that's enough. Do I need an array of servers with a total of 5Tb of RAM?

share|improve this question
    
I've never used Cassandra so I don't know about its redundancy and storage options but it seems to me that Cassandra would require X times the data storage for X copies. –  DerfK Dec 23 '12 at 16:49

3 Answers 3

up vote 5 down vote accepted

If you spread evenly your 5 TB of data on you 5 servers, each server will host 1 TB of data. Because of compaction needs, each server will need 2 TB of disk space (in the worst case, a compaction needs twice as much space on disk as you have data), which means 10 TB total in your cluster.

The case above is where you only store a single replica of your data among the cluster. In this case, if a server fail, one fifth of your data will be unreachable. If you want to store 2 replica of your data in the cluster, each node will need 4 TB of disk space, which means 20 TB total in your cluster.

share|improve this answer
    
That v straightforward. TY. Any indications on the amount of RAM? –  ethrbunny Dec 23 '12 at 20:46
1  
RAM is more complicated: not enough is never good, too much for Java and you will have large GC time. You should monitor your vm and gives it just what it need t avoid too long GC, and leave as much RAM as possible for disk cache. –  jfgagne Dec 24 '12 at 9:32

Supposing that all data is in RAM, that's a huge and expensive machine. Fortunately in most applications, you don't need to keep all your data in memory but only your live / active data.

Cassandra is able to retrieve automatically data from HD to RAM and back when the entry is missing from memory or getting "cold" respectively. You might compare Cassandra as a application level cache, where entries are account rows and cache hits/misses are referring to data already in memory or on the cassandra filesystem on disk.

So in terms of sizing:

  • HDD is dimensioned for the total dataset
  • RAM is dimensioned in order to reduce "cache misses"

So talking about RAM: You only need to keep enough data in memory in order to avoid unecessary refetches from disk. It's very application depended. I would suggest to run some benchmarking to veryfy how many many active sessions you get per day wrt to the total amount of sessions you have stored on the system. This applies well if your system is read dominated and reads have high locality. Check also this thread for further inspiration http://stackoverflow.com/questions/4924978/cache-design-question

This ratio of live data vs total data determines the RAM requirements for your system. Essentially, it's a tradeoff of cassandra misses vs RAM costs. Similar considerations - at a different level - apply to cpu cache design.

share|improve this answer

The cassandra suggested disk space per node for performance is 1/2 Terabyte so unless you want to wait for extremely long compactions and very long map/reduce times you should rethink how many machines are necessary.

share|improve this answer
    
Gag - seriously? Where can I read more about this? –  ethrbunny Dec 24 '12 at 5:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.