Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a large directory of log files on a server. All the logs are in one big directory.

We would like to back these up to another host (our archive server), and only keep the last 90 days of backups on the 'live' server.

however, due to the sheer number of logs that we generate with this application, we would create a massive folder that would be hard to browse. Is it possible, with RSYNC to specify the destination folders based on the date of the file (not today's current date)..

For example..

rsync <options> /logs/*.log ArchiveServer:/archive/ServerA/logs/<YYYY>/<MM>
where application.20111230.log gets dumped to /archive/ServerA/logs/2011/12
and application.20120301.log gets dumped to /archive/ServerA/logs/2012/03

I could dump them all into a generic folder, like /archive/ServerA/Current/ and then, maybe weekly look for files older than 90 days and sort them then.. but doing it directly in rsync seems like it would be much cleaner (and I wouldn't have two places to look for logs depending on how new it is)

share|improve this question
add comment

1 Answer

up vote 4 down vote accepted

Something like this?

rsync /logs/*.$(date '+%Y%m%d').log ArchiveServer:/archive/ServerA/logs/$(date '+%Y')/$(date '+%m')/

Update:

Right, so the date is already in the log filename. I should have read more slowly.

I don't think you would do this directly with rsync, but would wrap things up in a script. If you want to bang things through in a first run, something like:

for daysago in `seq 0 90`
do
  rsync /logs/*.$(date '+%Y%m%d' -d "$daysago days ago").log ArchiveServer:/archive/ServerA/logs/$(date '+%Y' -d "$daysago days ago")/$(date '+%m' -d "$daysago days ago")/
done

You can stick an "echo" in front of the rsync command in the loop to see if that's what you want.

Whatever you do will incorporate those elements, though other people will be more clever and elegant about the shell scripting than me.

share|improve this answer
    
doesn't that just get the current date, and format it to Y:M:D? or will it actually grab the date from the file name? –  Brian Mar 19 '12 at 15:10
    
@Brian Updated with a wrapper for the rsync –  cjc Mar 19 '12 at 15:26
    
that is awesome! thank you so much for the tips.. I think that will work perfectly.. (and easy to add and remove subdirectories as required.) –  Brian Mar 19 '12 at 15:48
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.