Make cron log rsync only if there was transfer

Question

I have this crontab running every 5 minutes:

date >> ~/18/rsync.log
rsync -vaz user@r18:~/assets ~/18 >> ~/18/rsync.log 2>&1

It adds this to the log file every 5 minutes:

Thu Aug 16 13:00:01 MSK 2012
receiving incremental file list

sent 506 bytes  received 541488 bytes  361329.33 bytes/sec
total size is 12954651209  speedup is 23901.84

And sometimes it adds actual transfer logs:

Thu Aug 16 13:10:01 MSK 2012
receiving incremental file list
assets/response/20120816/
assets/response/20120816/1017161.doc
assets/response/20120816/1017162.doc
assets/response/20120816/1017163.doc

sent 568 bytes  received 561686 bytes  1124508.00 bytes/sec
total size is 12954864201  speedup is 23040.95

I would like to omit empty transfer logging and keep actual transfer listings. Is there any way to configure rsync to produce verbose output only on non-empty transfers?

Answer 1

Don't use the -v option of rsync, instead use --out-format:

rsync --out-format="%n%L" -az user@r18:~/assets ~/18 >> ~/18/rsync.log 2>&1

Will only have output when you are transfering files, and no output when you don't. To have more fancy output look at the rsync.conf man page, in the "log format" section.

Answer 2

Please check rsync --log-file-format and --log-file switches. Default log file format adds 2 lines to log file even if not transfer anything, but please check manual. Maybe if you change log format will add only entry with transfered files.

Answer 3

Okay, I wrote some ingenious shell script to filter senseless spam coming from rsync on empty transfers. If you know any better way to detect it, please add your answer.

#!/bin/sh

LOG=$HOME/18/sync.log
TMP=$HOME/18/temp.log
SRC=user@r18:~/assets
DST=$HOME/18

echo >> $TMP
date >> $TMP
rsync -az $SRC $DST --log-file=$TMP --log-file-format='%10l %n%L'
[ `cat $TMP | wc -l` != 4 ] && cat $TMP >> $LOG
rm $TMP

Answer 4

You had better use Watcher to run rsync whenever a file changes. It based on incron but can monitor directories recursively.

$ git clonet git://github.com/splitbrain/Watcher.git
$ cd Watcher
$ cp watcher.ini ~/.watcher.ini

~/.watcher.ini

[DEFAULT]
logfile=/tmp/watcher.log
pidfile=/tmp/watcher.pid
[job1]
watch=/home/quanta/x
events=create,delete
recursive=true
autoadd=true
command=rsync -vaz quanta@localhost:~/x ~/y --log-file=/home/quanta/rsync.log

Starting the daemon:

$ ./watcher start
$ ps -ef | grep [w]atcher
quanta    3695     1  0 17:01 ?        00:00:00 /usr/bin/python2.7 ./watcher.py restart

rsync.log

2012/08/16 17:01:42 [3710] receiving file list
2012/08/16 17:01:42 [3724] .d..t...... x/
2012/08/16 17:01:42 [3724] >f.st...... x/a.txt
2012/08/16 17:01:42 [3724] sent 42 bytes  received 180 bytes  444.00 bytes/sec
2012/08/16 17:01:42 [3724] total size is 45  speedup is 0.20
2012/08/16 17:01:42 [3731] receiving file list
2012/08/16 17:01:42 [3745] sent 14 bytes  received 103 bytes  234.00 bytes/sec
2012/08/16 17:01:42 [3745] total size is 45  speedup is 0.38
2012/08/16 17:01:42 [3752] receiving file list
2012/08/16 17:01:42 [3766] sent 14 bytes  received 103 bytes  234.00 bytes/sec
2012/08/16 17:01:42 [3766] total size is 45  speedup is 0.38
2012/08/16 17:01:42 [3773] receiving file list
2012/08/16 17:01:42 [3787] sent 14 bytes  received 103 bytes  234.00 bytes/sec
2012/08/16 17:01:42 [3787] total size is 45  speedup is 0.38