I would like to create a new standard of two things:
how long would be generated and be rotated the logs generated by my applications ?
how to transfer the logs to Amazon S3, as a backup server ?.
I was thinking in use logrotate, to rotate and compress my daily files this way:
{filename}-{year}-{month}-{day}-{r-months}.gz
The r-months
variable means remain-months
, for how many months the file should remain in S3, files older than that should be removed.
A friend of mine, give the idea I should compress the logs daily
(in the new format proposed above) , after that these files should be sent to our bucket in Amazon S3.
Then files older than 7 days should be removed by logrotate
(cause they are in S3 already).
Nowadays, our applications use log4j
and others to generate logs.
1) Should we disable the versions logs, generated by our application and handle only with logrotate ?
2) In your opinion did you think this could crash some application ?
3) This new format of log, is a good one ?
4) And how send the files to S3 ? Now, I'm using s3cmd
, did you recommend me another tool ?