Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am tasked with trying to find a way to download zipped logs from a remote server. There are quite a bit of these logs and they are constantly created.

I do have limited ssh access to the remote server and can scp or rsync the files.

However, due to the sheer size of these logs file, I do not want to rsync all of them. The logs could get to terabytes and for rsync to compare them may take some time. I only want to get any new file that was created/last updated an hour ago.

I also am worried that I will rsync logs that are in the process of being created, so I was thinking to only rsync files that were last modified 3-5 minutes ago.

Would anyone be so kind as to help me with such a process? Thank you in advance.

share|improve this question
add comment

2 Answers

You can use ls -t to list the latest files by date and send that to rsync, you can easily set it to only transfer yesterdays logs or do you need it from the same day aswell?

share|improve this answer
add comment

Something like http://www.commandlinefu.com/commands/view/1481/rsync-find should do; you may want to execute the one-liner on the remote server and make it connect back to your "upload" server.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.