Logstash isn't send info to elasticsearch

Question

I have a Kibana environemtent installed, but isn't working.

When I try to configure kibana, kibana says: Couldn't find any Elasticsearch data

When I list indexes of elasticseach http://localhost:9200/_cat/indices?v it says: health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

I tink logstash is ready but isnt working properly, this is my logstash configs

/etc/default/logstash

JAVACMD="/usr/bin/java"
LS_HOME="/usr/share/logstash"
LS_SETTINGS_DIR="/etc/logstash"
LS_PIDFILE="/var/run/logstash.pid"
LS_USER="root"
LS_GROUP="root"
LS_GC_LOG_FILE="/var/log/logstash/gc.log"
LS_OPEN_FILES="16384"
LS_NICE="19"
SERVICE_NAME="logstash"
SERVICE_DESCRIPTION="logstash"

/etc/logstash/conf.d/logstash.conf

  input {
      file {
        path => "/var/log/apache2/access.log"
        start_position => "beginning"
      }
    }

    filter {
      if [path] =~ "access" {
        mutate { replace => { "type" => "apache_access" } }
        grok {
          match => { "message" => "%{COMBINEDAPACHELOG}" }
        }
      }
      date {
        match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
      }
    }

    output {
      elasticsearch {
        hosts => ["localhost:9200"]
      }
    }

Error in syslog:

[main] Logstash - java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (OpenSSL::X509::StoreError) setting default path failed: the trustAnchors parameter must be non-empty

How can I solve this problem? any Idea about what is happening?

Answer 1

Your diagnosis is correct, ElasticSearch is running but there are no indexes created. This is why you get this:

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
[whole lot of nothing]

What you should should be getting is a .kibana index in there, which Kibana makes the moment it connects to an ElasticSearch system.

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
  }
}

Is what that should like. The http:// tells Logstash to connect to the ElasticSearch on the localhost using an unencrypted connection. In theory using a bare hostname:port should get you that, but using http:// will make sure of it. And also make sure that LogStash won't mobilize the x509 infrastructure for connecting to its one output.

Also take a look at your ElasticSearch logs to make sure they're playing nice. Sometimes if it takes exception to what Logstash is feeding it you will see lots of bombs in there. Those can be diagnostic all their own.

You should make sure your kibana.yml file also uses the same http://localhost:9200 setting for elasticsearch.url.

SSL is a paid-for feature from X-Pack in ElasticSearch. It's very easy to turn on and have not work, alas.

Answer 2

For java version 10.x.

sudo rm /etc/ssl/certs/java/cacerts
sudo update-ca-certificates -f 
sudo /var/lib/dpkg/info/ca-certificates-java.postinst configure 

For x-pack ...
try

output {
  elasticsearch {
    hosts => ["https://localhost:9200"]
  }
}

Kind regards, Vadim.