We perform nightly backups of a Windows file server by first creating an incremental backup file on the server each night (as well as a complete backup on Thursday night), and then copying that to a backup server running Linux/Ubuntu. In order to maintain off-site redundancy, we then rsync the backup directory to an external drive which is rotated after each nightly run.
Over time, the number of incremental backup files has increased at a steady pace (although, the size of these files has varied).
We've also noticed that the rsync process has steadily been taking longer, even though it is only copying the latest two files to the external drive.
This is the command:
rsync -vr --delete-before --log-file=/rsync_log.csv /backup/archive/ /mnt/usbdisk/ 2>&1
When we tested the command, and investigated the log file, we saw this:
... 2011/02/14 23:59:35  >f..T...... IncrementalBackup_2011_02_06_20.bkf 2011/02/15 00:00:45  >f..T...... IncrementalBackup_2011_02_08_20.bkf 2011/02/15 00:03:22  >f..T...... IncrementalBackup_2011_02_09_20.bkf 2011/02/15 00:04:36  >f..T...... IncrementalBackup_2011_02_11_20.bkf 2011/02/15 00:04:51  >f..T...... IncrementalBackup_2011_02_12_20.bkf 2011/02/15 00:05:06  >f..T...... IncrementalBackup_2011_02_13_20.bkf 2011/02/15 00:06:13  >f+++++++++ IncrementalBackup_2011_02_14_20.bkf 2011/02/15 00:54:32  >f..T...... Thursday_Full_Backup_2011_01_20.bkf 2011/02/15 03:24:41  >f..T...... Thursday_Full_Backup_2011_01_27.bkf ...
What we found was the time taken on each file related to the size of the file - even when skipping it (example - the full backup took about 2.5 hours to process, while the incrementals about 2-3 minutes or less).
The only file actually copied is the latest incremental file.
The only explanation we can think of is that rysnc is performing a checksum of the file - even though the documentation says it does not by default, and we have not specified the --checksum switch on the command. Surely it can't take 2.5 hours to determine the timestamp and filesize?
After having gone over the documentation, I can't find any other explanation than the checksum is being calculated. So, is there a way to be sure that checksum is disabled?