Since transferring or copying a file that is being used sometimes causes corruption of the transferred file, can we define a time interval in which Rsync checks each file in a given directory to see if there is a change within that time interval ?

Files that are not changed during that interval will be transferred, while those that have changes will not.

Can I do that with rsync ? Or another tool ? Is there a script to add this functionality to Rsync ?



just do it again.

seriously, the first time, most of the files will be transferred perfectly, and those that were in flux while copying will still have most of the data correctly. the second time, rsync will copy only the differences, in effect just correcting what went wrong.

what i usually do is to copy once without stopping any process, this can take hours, or even days. then do it again, it might take several minutes. so i do it again, this time it's as low as it will be. then i stop the services, copy one last time, and restart the service. downtime is usually under 10 minutes.

of course, this can only be done for services that can be stopped. for really unstoppable things, i use LVM snapshots. way more complex and slow, but guaranteed 'point in time' copy.

  • Makes sense, but is there a way to define a time interval to checks each file in a given directory ? In my case, calculations are made on these files. Thanks – user44782 Jun 4 '10 at 15:04
  • Ok, I can search for files that have been modified in the last 24 hours, from the current working directory and all subdirectories, by the following command: find . -ctime 0 -type f – user44782 Jun 4 '10 at 20:31
  • why you need to know which files have changed? rsync won't copy what's already on the target. – Javier Jun 7 '10 at 3:30
  • My worflow is : move files to a new file-system, a script check this new file-system, and make calculations based on those files. But I have to be sure I'm working on good files (generating source files can take few hours), and I can have to work on multi-files (I need to check I get all files before running calculations). – user44782 Jun 7 '10 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.