For certain things, I want to hold onto logs for a long time - for example my apache logs for historical interest. But even there, I have a
cron job running every day and/or week to do an simple analysis of unique visitors that gets mailed to a gmail account I established just for that sort of thing.
However, my general approach is that I don't want or need most of the data in those logs going back more than a few days.
I already know that I'm never going to "get around tuit" when it comes to doing any graphing or historical analysis because, frankly, I'm too busy doing my "real" job :)
If you're running a
syslog collector, you may need to hold onto those logs longer - just because they're grabbing everything from however many servers you're collecting from.
The last time I had a
syslog server setup, we had a pair of old DL180s with 18GB harddrives running Ubuntu. Both cross-mounted the other via nfs (
<othersys>/path/to/log @ <currentsys>/path/to/backup).
We rotated our logs daily, compressing via
bzip2. When the drive space hit >90% used, we'd drop the oldest file.
It's been mentioned before*, but you may also want to investigate a log analyzer such as epylog or Splunk as a component of your log policy.