0

I'm setting up a ELK cluster using Centos 8 and version 7.4 of Elasticsearch, Logstash and Kibana. My issue is with Logstash not picking up the events coming through syslog. Configuring Logstash to read from files and sending it to elasticsearch it works fine.

An overview of the infra, the 3 systems are installed in this server.

I have SELinux disabled to make sure it's not affecting it. Logstash config is using port 5144 and I can see that it's working fine with:

[root@elk-1 conf.d]# netstat -tulpn | grep 5144
udp        0      0 0.0.0.0:5144            0.0.0.0:*                           15838/java

For firewall rules I'm using the following (firewalld):

firewall-cmd --set-default-zone=internal
firewall-cmd --permanent --zone=internal --add-port=514/tcp #syslog port
firewall-cmd --permanent --zone=internal --add-port=514/udp #syslog port
firewall-cmd --permanent --zone=internal --add-port=5514/tcp #syslog forwarded port
firewall-cmd --permanent --zone=internal --add-port=5514/udp #syslog forwarded port
firewall-cmd --permanent --zone=internal --add-port=5600/tcp #kibana
firewall-cmd --permanent --zone=internal --add-port=5601/tcp #kibana
firewall-cmd --permanent --zone=internal --add-port=9600/tcp #logstash
firewall-cmd --permanent --zone=internal --add-port=9200/tcp #elasticsearch
firewall-cmd --permanent --zone=internal --add-port=9300/tcp #elasticsearch
firewall-cmd --permanent --zone=internal --add-port=80/tcp #http
firewall-cmd --permanent --zone=internal --add-port=443/tcp #https
firewall-cmd --zone=internal --add-forward-port=port=514:proto=udp:toaddr=127.0.0.1:toport=5514 --permanent
firewall-cmd --zone=internal --add-forward-port=port=514:proto=tcp:toaddr=127.0.0.1:toport=5514 --permanent
firewall-cmd --zone=internal --add-masquerade --permanent

So I'm redirecting the traffic from 514 to 5144. I can see the traffic comming through port 514 with tcpdump. However I don't see any traffic on port 5144. Checking the Logstash logs doesn't have anything helpful even on debug mode.

The input section of my logstash is:

input {
  udp {
    port => 5144
    type => syslog
  }
}

and output:

output {
  elasticsearch {
    hosts => [ "10.248.1.31:9200" ]
    manage_template => false
    index => "fgt-%{+YYYY.MM.dd}"
    }
}

* UPDATE *

Setting Logstash to use root instead of logstash user and the config file to listen to 514 it worked. However I know that's not the best idea and would like a better fix for that.

Any ideas on why Logstash is not picking up the traffic from syslog ?

Thanks

5
  • 1
    Have you tried echoing log lines (or any text really) directly to the ports in question with nc or telnet? Something like nc localhost 5144 for TCP, or nc --udp localhost 5144 for UDP.
    – GregL
    Nov 20 '19 at 13:03
  • Hi Greg, yeah in fact I did: echo 'Hellow World' | nc --udp elk-1.ammeon.com 5144 and with 514. I can see that in tcp dump, but not on Kibana. So logstash is not sending it somehow. Also, if I do 514 and do a tcpdump on 5144 I don't see anything. I would expect to see since it's being redirect right ?
    – Adonist
    Nov 20 '19 at 15:32
  • So nothing is showing up in Kibana even when you NC it directly to 5144? Have you run LS with the stdout output, to see if your pipeline is working? I've also found that the syslog type for inputs doesn't work as well as I'd like. I'm using TCP, but I don't have a type set.
    – GregL
    Nov 20 '19 at 15:47
  • I've updated the message. It didn't work. setting for stdout wasn't working either. I configured logstash to run as root and now it worked, however that's not the best solution anyway.
    – Adonist
    Nov 20 '19 at 16:03
  • 1
    Perhaps try a plain UDP input with a stdout output, to see if you can narrow down the cause of the issues. Also, run Logstash interactively while running through the config iterations so that you can see what it's doing each time.
    – GregL
    Nov 20 '19 at 16:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.