Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a directory that has many folders with code inside it, and I need to run a script to rsync changes over to a couple other servers. I have that script written and working fine, but I cannot find something that will check the directory tree for changes. I tried stat, but that only check one folder down, not recursively (as far as I could figure out). Anyone know of a command or program I could install to do this? I use Ubuntu 10.04. Thanks in advance.

share|improve this question
add comment

6 Answers

inotifywait will do recursive checks.

It's in the Ubuntu repositories.

share|improve this answer
add comment

The perfect tool for this job is using the inotify kernel service. You can use them on a shell script with the package inotify-tools (debian/ubuntu). You can read more and see some examples on the project's page. There's even an example on the page that seems to do something close to what you want:

#!/bin/sh

# get the current path
CURPATH=`pwd`

inotifywait -mr --timefmt '%d/%m/%y %H:%M' --format '%T %w %f' \
-e close_write /tmp/test | while read date time dir file; do

       FILECHANGE=${dir}${file}
       # convert absolute path to relative
       FILECHANGEREL=`echo "$FILECHANGE" | sed 's_'$CURPATH'/__'`

       rsync --progress --relative -vrae 'ssh -p 22'  $FILECHANGEREL usernam@example.com:/backup/root/dir && \
       echo "At ${time} on ${date}, file $FILECHANGE was backed up via rsync"
done
share|improve this answer
1  
The problem here is if the files are actively changing and rsync takes significant time to complete, an unrecoverable backlog could be created. See my answer here and the attached comments. Great care should be taken that rsync completes quickly or a while inotifywait ... loop (without the -m switch) should be used. Avoiding the temptation to background the rsync should be carefully considered. –  Dennis Williamson Jul 20 '10 at 19:13
    
Hear what Dennis is saying, its wise advice. Maybe if he makes a script that accumulates changes and then rsync then from time to time to prevent multiple processes... –  coredump Jul 20 '10 at 21:14
add comment

rsync will check the directory tree for changes and copy only the changed files if you give it a directory as the source.

share|improve this answer
    
True, but what if there are new enough files that it takes more than a minute to copy? I also want this to be able to run as soon as changes happen, not a minute later. This copies code to live webservers, and having them out of sync for a minute is not ideal. –  Mike Jul 20 '10 at 17:26
    
Before I learned about inotify and inotifywait I'd have said if that happens split the rsync source directories. Inotify is clearly the way forward though. –  Iain Jul 20 '10 at 19:04
add comment

Have a look at inotify. I solved a similar problem some time ago with dnotify or sgi_fam (don't recall which) but inotify seems to have superceeded them both.

share|improve this answer
add comment

Someone already invented this wheel. It's called lsyncd and it is apparently a mashup of rsync and inotify. I haven't used it myself, because the limitations of inotify are such that it doesn't scale to really large (and deep) directories. I'm waiting until they reimplement it directly using fsnotify.

There's also PIrsyncD which seems to do the same thing.

share|improve this answer
add comment

use find

find /path -mmin 1

That will check if a file or directory was modified 1 min ago, look at the man page for more options.

share|improve this answer
    
Is there a way to write this script so that basically if that command returns anything, then run my rsync script, else sleep for a minute? –  Mike Jul 20 '10 at 17:11
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.