I'm building a log analyser service to start monitoring mainly our pfSense Firewalls, XenServer Hypervisors, FreeBSD/Linux servers and Windows servers.

There's a lot of documentation on the internet about the ELK stack and how to make it work nicely. But I would like to use it in a different manner, but I don't know if it's a good solution or just a waste of time/disk space.

I already have a FreeBSD 10.2 machine acting as a remote syslog server, and my ideia is to simply concentrate all the logs on this machine and them the syslog server forwards the logs with logstash-forwarder to the ELK server.

It's clear to me that this approach will raise the disk requirements for this setup, but in other hand I will have only one machine with the logstash-forwarder daemon installed, which seems good to me.

But talking about problems. The logstash parser matches [host] with the hostname of the server sending the log messages, and in this approach there's only on "server" show on ELK, the remote syslog server.

I'm aware that I can customize the settings on the logstash configuration files but I don't know (and I don't have the experience to know) if this is just a simple setting on the parsers of it if will compromise the entire ELK experience.

In the end I just want some advices about my logging architecture and if it will work, or if I should go without other option.

Thanks in advance,

  • I'm sure what you want to do is possible, but details about what format you're logging in on the central syslog server, the logging rate, etc would be great. Also, asking a specific question will get you much better responses than simply saying "I'm lost, please help". – GregL Nov 24 '15 at 1:42
  • Have you considered graylog? You can use the graylog syslog input to ingest syslog into elasticsearch. From there you can use Kibana and graylog has a syslog/pfsense extractor available. – davey Nov 24 '15 at 6:28
  • Logstash also has a syslog input... – GregL Nov 24 '15 at 12:51

Yes. It is possible to change the host field in logstash output with ruby filter without much hassle.

    ruby {
            code => "
                    event['host'] = event['message'].split(' ')[3]

Here I assumed in the syslog server logs, the host field is the fourth field where white space is the separator.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.