Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am using inotify to watch a directory and sync files between servers using rsync. Syncing works perfectly, and memory usage is mostly not an issue. However, recently a large number of files were added (350k) and this has impacted performance, specifically on CPU. Now when rsync runs, CPU usage spikes to 90%/100% and rsync takes long to complete, there are 650k files being watched/synced.

Is there any way to speed up rsync and only rsync the directory that has been changed? Or alternatively to set up multiple inotifywaits on separate directories. Script being used is below.

UPDATE: I have added the --update flag and usage seems mostly unchanged

#! /bin/bash


inotifywait -e "$EVENTS" -m -r --format '%:e %f' /var/www/ --exclude '/var/www/.*cache.*' | (
while true; do
    read -t 1 LINE;
    if test -z "$LINE"; then
        if test ! -z "$WAITING"; then
                echo "CHANGE";
                rsync --update -alvzr --exclude '*cache*' --exclude '*.git*' /var/www/* root@secondwebserver:/var/www/
share|improve this question
up vote 0 down vote accepted

I ended up removing the compression option (z) and upping the WAITING var to 10 (seconds). This seems to have helped, rsync still spikes CPU load but it is shorter lived. Credit goes to an answer on unix stackexchange

share|improve this answer

You could also look into lsyncd which has more options. You can use lua scripting, so then say how many concurrent processes it needs to use and perhaps more things that might be of interest.

This uses rsync and ionotify too.

share|improve this answer

You may also consider to disable SSH compression and using a less demanding encryption algorithm, example:

rsync -e "ssh -c arcfour -x -o Compression=no" ....

One other measure can be to setup your ssh tunnel explicitly. You need to setup an Rsync daemon for that purpose though. Check the shell script at our FAQ. It should be usable for your case as well.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.