Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have the logs traditionally kept in /var/log/ piped through syslog-ng. The logs can reach to the terabyte size on a daily basis. In order to make them more manageable. I would like break them out by server and app cluster.

Is this the best way to do it? How do I do this with Syslog-ng that is manageable?

share|improve this question
3  
that's a LOT of log traffic - out of curiosity, what are you logging? –  voretaq7 Oct 7 '11 at 20:46
    
A few thousand servers and related applications. –  Thomas Vincent Oct 11 '11 at 23:48
add comment

2 Answers

up vote 4 down vote accepted

You can do this with syslog-ng, and it's one of the most common ways to break up logs. Please refer to the syslog-ng manual for information, as well as the destination configured in this guy's sample config file (the big hint I'll give you is you want to generate either the log path or the filename using the hostname of the sending server as a component).

Is this the "best" way to do it? There is no universal "best".
This is certainly viable - I managed an entire ISP's logging infrastructure with this sort of breakdown, though it didn't have the volume you're talking about.
If it makes sense for you to structure your logs this way and it keeps the log sizes manageable than this may be the "best" way for your environment.

share|improve this answer
add comment

You should really consider building a better syslog "management" solution (as opposed to just monitoring). There's a really good whitepaper posted on Cisco's website that outlines methods (including syslog-ng) as well as processes and tools: http://www.cisco.com/en/US/technologies/collateral/tk869/tk769/white_paper_c11-557812.html

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.