1

I don't remember changing anything at all on my logstash server. As of just yesterday it was working fine! And I used it to run some queries.

However.. today when I went to my logstash page, I see only thing message come up:

Connection Failed

Possibility #1: Your elasticsearch server is down or unreachable

This can be caused by a network outage, or a failure of the Elasticsearch process. If you have     recently run a query that required a terms facet to be executed it is possible the process has run out     of memory and stopped. Be sure to check your Elasticsearch logs for any sign of memory pressure.

Possibility #2: You are running Elasticsearch 1.4 or higher

Elasticsearch 1.4 ships with a security setting that prevents Kibana from connecting. You will need     to set http.cors.allow-origin in your elasticsearch.yml to the correct protocol, hostname, and port (if     not 80) that your access Kibana from. Note that if you are running Kibana in a sub-url, you should     exclude the sub-url path and only include the protocol, hostname and port. For example,     http://mycompany.com:8080, not http://mycompany.com:8080/kibana.

Click back, or the home button, when you have resolved the connection issue

And I can easily tell that elasticsearch 1.1.1 is running on that host:

[root@logs:~] #ps -ef | grep elasticsearch | grep -i -v -e grep -e screen

root     16666  9640  6 09:00 pts/1    00:05:49 /etc/alternatives/javahome/bin/java -Xms256m -Xmx1g     -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -    XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError     -Delasticsearch -Des.foreground=yes -Des.path.home=/usr/local/elasticsearch-1.1.1 -cp     :/usr/local/elasticsearch-1.1.1/lib/elasticsearch-1.1.1.jar:/usr/local/elasticsearch-    1.1.1/lib/*:/usr/local/elasticsearch-1.1.1/lib/sigar/* org.elasticsearch.bootstrap.Elasticsearch

And I can see, in fact that elastic search is listening on the right ports:

[root@logs:~] #netstat -tulpn | grep -i listen | grep java

tcp        0      0 :::2541                     :::*                        LISTEN      16722/java

tcp        0      0 :::9200                     :::*                        LISTEN      16666/java    

tcp        0      0 :::9300                     :::*                        LISTEN      16666/java

tcp        0      0 :::9301                     :::*                        LISTEN      16722/java

All of the above ports are for elasticsearch. I use port 2541 for logstash. That's fine too.

And logstash is doing it's thing as well. I usually leave logstash running in verbose tucked away in a screen session. And when I go to that screen session I can see all the log information flying by.

I've tried stopping and starting both elasticsearch as well as logstash. But I still see the same message.

Can someone please let me know what's going on? How do I correct this?

Thanks

  • With the information you provided I presume ElasticSearch is running low on memory and has stopped functioning properly. Did you try to restart it? Did you check ES logs? Maybe you should try and increase Xmx for ES. – Jakov Sosic Nov 25 '14 at 0:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.