Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

We know logs are important. What is the best practice to maintain them?

Logs can get big - so we go for log rotation. And sure we don't want to lose the logs - so we go for their backups.

What is the best practice of handling these things? please advise.

share|improve this question

closed as not constructive by ewwhite, Zoredache, Khaled, Scott Pack, MDMarra Nov 14 '11 at 2:22

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

It depends on you requirements/environment. There is no one true 'best'. – Zoredache Nov 5 '11 at 18:51
Are hoping for a general answer on all possible logfiles that a system produces? – faker Nov 5 '11 at 18:53
up vote 4 down vote accepted

Logrotate is a good tool for log rotation and compression. It is available for most Unix flavors. I find the defaults used on Ubuntu to be a good starting point. When changing rotation frequency, you should change the rotate count.

Keep logs as long as they may be useful. Archive to non-disk storage if necessary. For business systems there may be legal requirements directing retention and/or destruction of log data.

Depending on the data and availability of data in another log data may be kept for a week, a month, a quarter, or a year. Only duplicated data is deleted after a week. A monthly backup retained for a year will give you a years worth of most log data.

A centralized log server generally serves a different function than log retention. Comparing the centralized logs to host logs may detect log file tampering.

share|improve this answer

I think BillThor provides a good answer. As he said, centralized log collection is a good idea. If you put your logs into a database you can do all sorts of on the fly queries, which can be tremendously helpful. One useful open source log management tool is logstash. Commerical tools such as splunk serve the same purpose and might be appropriate if you want more formal support.

Something else to consider about managing logs on individual servers: log rotation has two components: rotating on a schedule, and rotating based on file size. Your log rotation should take both in to account. If you are just rotating a log file on a weekly basis and something suddenly starts logging huge amounts of data, you are going to run out of disk space. Similarly, if you are just rotating on file size you could quickly lose old logs if the log volume dramatically increases. This is another excellent reason to consider centralized log management.

share|improve this answer

What environment?

On Linux logrotate is a good tool.

Centralised logging with syslog is a good practice too.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.