Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I use Logstash and Elasticsearch for our squid log store and analyze.The size of the logs increase at the speed of 40Gb per day in our freeBSD ZFS storage system. the Elasticsearch gets into failure every five days and no logs can be further written.

And I tried

index.number_of_shards: 1
index.number_of_replicas: 0

But it seems no use.

attached is the snapshot of the plugs head

elasticsearch head plug status

Can anyone explain what I'm doing wrong? And which configuration should I modify.

UPDATE

The log

[2013-12-08 19:51:16,916][WARN ][index.engine.robin ] [David Cannon] [logstash-2013.12.08][0] failed engine java.lang.OutOfMemoryError: Java heap space

[2013-12-09 17:03:07,500][DEBUG][action.admin.cluster.node.info] [David Cannon] failed to execute on node [sSoHeIz5TSG8fR3IRHj_Pg] org.elasticsearch.transport.RemoteTransportException: Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.node.info.NodeInfo] Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.node.info.NodeInfo]

share|improve this question
    
Can you post some error logs –  pradeepchhetri Dec 10 '13 at 9:25
    
Sorry I add the log there.But there are so many logs so I paste some maybe help. –  Kindule Dec 10 '13 at 9:34
    
What are the specs of the machine running Logstash? –  Tom O'Connor Dec 10 '13 at 11:04
    
The Logstash running on the same machine on the Elastic-search. –  Kindule Dec 10 '13 at 22:18
add comment

1 Answer

up vote 1 down vote accepted

I've seen this before, and it's all to do with the amount of Heap Space allocated to Logstash by the JVM.

You can try increasing this by providing this flag -XX:MaxHeapSize=256m to the JVM when you start it, except you should probably try setting the MaxHeapSize to something like 512m or even bigger.

Elasticsearch comes with some pretty sane defaults, but it's possible to do even more tuning, setting the size of it's heap and so on for searching.

ES and Logstash can scale to billions of events, and many terabytes of log data, with careful configuration.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.