1

the main requirement I have is to be able to view and do simple searches on logs combined from multiple machines. However, I would like the solution to have minimal impact on the rest of the (core) system. I don't have any real-time requirements, the process can be asynchronous.

Initially syslog seemed like a good option, but what if the syslog server dies? In the worst case the core system's user see errors, in the best case some logs are lost.

So I started looking around and found Logstash (http://logstash.net/). Currently my idea is:

  • on each server (that's running the system's core components) there is a Logstash agent running
  • the agent monitores log files and sends them to an ElasticSearch cluster
  • there's another server with the Logstash UI

That way:

  • there's no single point of failure
  • even if the ES cluster dies, only the agents will be affected - the application will still happily write logs to files
  • after ES comes back, the agent will (hopefully) catch up and send all the pending logs (is Logstash smart enough to do that?)

Do you think this will work? Or maybe you could recommend another solution?

2 Answers 2

1

Rsyslog has some of the functionality that you are describing and the project also has plenty of documentation about reliable message forwarding.

In short, with rsyslog you can use the RELP protocol for reliable syslog message forwarding and then you don't have to worry about message loss. And you also have the option of configuring local spool files, where rsyslog will buffer messages in case the remote server is down. Once the remote comes back up, your agent will catch up.

You also have the option of configuring rsyslog to write to a relational database, and then you can make the database as redundant as you want (I personally find a syslog server easier to cluster).

0
0

This question is most likely going to be closed as off topic, please see the FAQ.

Regardless, syslog (or any syslog based system) should work just fine, after all if you are concerned about losing logs then make sure to backup the syslog server as part of your normal DR scenario. It's a pretty simple task/request.

4
  • I took a look at the FAQ, seems on-topic to me :).
    – adamw
    Nov 6, 2012 at 21:45
  • Sorry, but no it is not. Or maybe you could recommend another solution? is a product and solution solicitation, check the FAQ again, that is not allowed on the site. I'm glad you found an answer but please double check to make sure your questions are about a specific problem or question in the future. Nov 6, 2012 at 21:58
  • Well, I wasn't looking for a product, sorry maybe the wording wasn't best, I'm not english-native :). But I would say that most of the questions look for "solutions" to problems, here looking for a fault-tolerant log collection. Anyway, answered, thanks for the help :).
    – adamw
    Nov 7, 2012 at 11:28
  • Yes, just be careful about how you word your questions in the future to avoid soliciting recommendations or potential solutions. Instead, identify one and then ask questions about it. If you do that, much less chance of someone (like me) voting to close the question. Nov 7, 2012 at 13:40

Not the answer you're looking for? Browse other questions tagged .