Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have the logs traditionally kept in /var/log/ piped through syslog-ng. The logs can reach to the terabyte size on a daily basis. In order to make them more manageable. I would like break them out by server and app cluster.

Is this the best way to do it? How do I do this with Syslog-ng that is manageable?

share|improve this question
that's a LOT of log traffic - out of curiosity, what are you logging? – voretaq7 Oct 7 '11 at 20:46
A few thousand servers and related applications. – Thomas Vincent Oct 11 '11 at 23:48
up vote 4 down vote accepted

You can do this with syslog-ng, and it's one of the most common ways to break up logs. Please refer to the syslog-ng manual for information, as well as the destination configured in this guy's sample config file (the big hint I'll give you is you want to generate either the log path or the filename using the hostname of the sending server as a component).

Is this the "best" way to do it? There is no universal "best".
This is certainly viable - I managed an entire ISP's logging infrastructure with this sort of breakdown, though it didn't have the volume you're talking about.
If it makes sense for you to structure your logs this way and it keeps the log sizes manageable than this may be the "best" way for your environment.

share|improve this answer

You should really consider building a better syslog "management" solution (as opposed to just monitoring). There's a really good whitepaper posted on Cisco's website that outlines methods (including syslog-ng) as well as processes and tools:

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.