I have the logs traditionally kept in /var/log/ piped through syslog-ng. The logs can reach to the terabyte size on a daily basis. In order to make them more manageable. I would like break them out by server and app cluster.

Is this the best way to do it? How do I do this with Syslog-ng that is manageable?

  • 3
    that's a LOT of log traffic - out of curiosity, what are you logging? – voretaq7 Oct 7 '11 at 20:46
  • A few thousand servers and related applications. – Thomas Vincent Oct 11 '11 at 23:48

You can do this with syslog-ng, and it's one of the most common ways to break up logs. Please refer to the syslog-ng manual for information, as well as the destination configured in this guy's sample config file (the big hint I'll give you is you want to generate either the log path or the filename using the hostname of the sending server as a component).

Is this the "best" way to do it? There is no universal "best".
This is certainly viable - I managed an entire ISP's logging infrastructure with this sort of breakdown, though it didn't have the volume you're talking about.
If it makes sense for you to structure your logs this way and it keeps the log sizes manageable than this may be the "best" way for your environment.


You should really consider building a better syslog "management" solution (as opposed to just monitoring). There's a really good whitepaper posted on Cisco's website that outlines methods (including syslog-ng) as well as processes and tools: http://www.cisco.com/en/US/technologies/collateral/tk869/tk769/white_paper_c11-557812.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.