Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am looking to scope the servers required for an elasticsearch proof-of-concept.

Ultimately, my question is this:

Given 1GB of json text indexed by elasticsearch, how much can I expect to be occupied on disk by elasticsearch?

Obviously there are many variables, but I'm going for orders of magnitude. 100MB? 100GB?

I understand that elasticsearch performs compression ( http://www.elasticsearch.org/guide/reference/index-modules/store/ ), but I don't know what kind of footprint the indexes and other structures occupy.

Anecdotal answers are acceptable, but please also let me know what version you're using.

share|improve this question
2  
150 MB, that's approx. –  BigSack Aug 26 '13 at 18:56
add comment

1 Answer 1

up vote 4 down vote accepted

The answer is: it depends.

A blog post by Adrien Grand, who works on Elasticsearch did some benchmarking with this from the Lucene standpoint. It looks like he was getting about 2x improvement.

He also mentions LUCENE-4226, where some of this underlying compression work was done. There's a few benchmarks listed within, comparing the various compression algorithms tested.

As well, based on this Elasticsearch 0.19.5 release announcement, it appears that store-level compression defaults to LZF, with Snappy coming some time in the future. Further looking around showed that Snappy experimental support appeared in 0.19.9

share|improve this answer
    
I neglected to search for Lucene in my research, which I should have. Links have been informative. –  cmonkey Aug 27 '13 at 16:46
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.