I'm trying to setup a central logstash configuration. However I would like to be sending my logs through syslog-ng and not third party shippers. This means that my logstash server is accepting via syslog-ng all the logs from the agents.

I then need to install a logstash process that will be reading from /var/log/syslog-clients/* and grabbing all the log files that are sent to the central log server. These logs will then be sent to redis on the same VM.

In theory I need to also configure a second logstash process that will read from redis and start indexing the logs and send them to elasticsearch.

My question:

Do I have to use two different logstash processes (shipper & server) even if I am in the same box (I want one log server instance)? Is there any way to just have one logstash configuration and have the process read from syslog-ng ---> write to redis and also read from redis ---> output to elastic search ?

Diagram of my setup:

[client]-------syslog-ng---> [log server] ---syslog-ng <----logstash-shipper ---> redis <----logstash-server ----> elastic-search <--- kibana

  • Why are you writing to redis if everything is already on the same server? – Dave Wongillies Sep 3 '13 at 11:37
  • Your setup only needs to be: [client]-------syslog-ng---> [log server] ---syslog-ng <----logstash ----> elastic-search <--- kibana – Dave Wongillies Sep 3 '13 at 11:48

I'm not sure if I understood the question correctly, but I do know that Syslog-NG can ship directly to Logstash without the need for an additional shipper as an intermediary. You could define a destination in syslog-ng.conf similar to this example:

destination d_logstash { 
  tcp("" port(5514)); 

And then define a log action to send Syslog messages from source s_src to destination:

log {

Which should enable the message transmission. Don't forget to restart the syslog-ng service to apply the changes.

source: The Logstash Book

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.