3

I have a directory that has many folders with code inside it, and I need to run a script to rsync changes over to a couple other servers. I have that script written and working fine, but I cannot find something that will check the directory tree for changes. I tried stat, but that only check one folder down, not recursively (as far as I could figure out). Anyone know of a command or program I could install to do this? I use Ubuntu 10.04. Thanks in advance.

6 Answers 6

3

inotifywait will do recursive checks.

It's in the Ubuntu repositories.

3

The perfect tool for this job is using the inotify kernel service. You can use them on a shell script with the package inotify-tools (debian/ubuntu). You can read more and see some examples on the project's page. There's even an example on the page that seems to do something close to what you want:

#!/bin/sh

# get the current path
CURPATH=`pwd`

inotifywait -mr --timefmt '%d/%m/%y %H:%M' --format '%T %w %f' \
-e close_write /tmp/test | while read date time dir file; do

       FILECHANGE=${dir}${file}
       # convert absolute path to relative
       FILECHANGEREL=`echo "$FILECHANGE" | sed 's_'$CURPATH'/__'`

       rsync --progress --relative -vrae 'ssh -p 22'  $FILECHANGEREL [email protected]:/backup/root/dir && \
       echo "At ${time} on ${date}, file $FILECHANGE was backed up via rsync"
done
2
  • 1
    The problem here is if the files are actively changing and rsync takes significant time to complete, an unrecoverable backlog could be created. See my answer here and the attached comments. Great care should be taken that rsync completes quickly or a while inotifywait ... loop (without the -m switch) should be used. Avoiding the temptation to background the rsync should be carefully considered. Jul 20, 2010 at 19:13
  • Hear what Dennis is saying, its wise advice. Maybe if he makes a script that accumulates changes and then rsync then from time to time to prevent multiple processes...
    – coredump
    Jul 20, 2010 at 21:14
2

rsync will check the directory tree for changes and copy only the changed files if you give it a directory as the source.

2
  • True, but what if there are new enough files that it takes more than a minute to copy? I also want this to be able to run as soon as changes happen, not a minute later. This copies code to live webservers, and having them out of sync for a minute is not ideal.
    – Mike
    Jul 20, 2010 at 17:26
  • Before I learned about inotify and inotifywait I'd have said if that happens split the rsync source directories. Inotify is clearly the way forward though.
    – user9517
    Jul 20, 2010 at 19:04
1

Have a look at inotify. I solved a similar problem some time ago with dnotify or sgi_fam (don't recall which) but inotify seems to have superceeded them both.

1

Someone already invented this wheel. It's called lsyncd and it is apparently a mashup of rsync and inotify. I haven't used it myself, because the limitations of inotify are such that it doesn't scale to really large (and deep) directories. I'm waiting until they reimplement it directly using fsnotify.

There's also PIrsyncD which seems to do the same thing.

0

use find

find /path -mmin 1

That will check if a file or directory was modified 1 min ago, look at the man page for more options.

1
  • Is there a way to write this script so that basically if that command returns anything, then run my rsync script, else sleep for a minute?
    – Mike
    Jul 20, 2010 at 17:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .