0

I have the following directory structure:

/tmp1
  \_ file1.txt
  \_ file2.txt

/tmp2
  \_ file1.txt
  \_ file2.txt
  \_ file3.txt
/changes

With rsync's --compare-dest option, I want to sync all files into /changes that are not present in /tmp1 but in /tmp2. Modified files in /tmp2 that differ from /tmp1 should also be synced to /changes.

When I execute the following command:

rsync -acvu --compare-dest=/tmp1/ /tmp2/ changes

file3.txt will be saved in changes. But also file1.txt although the md5 sum of the file in both directories is equal!

Did I miss some parameters for rsync? Thanks in advance!

UPDATE:

I find out that the files that shouldn't be synced have file permission -rw-rw-r-- others have -rwxrwxr-x. I can't find something about this behavior in relation to rsync.

Output of "ls -la" on both directories:

# ls -la /tmp1
-rw-rw-r-- 1 1001 1001 4 Dec 31 15:00 file1.txt
-rwxrwxr-x 1 1001 1001 6 Dec 31 15:00 file2.txt

# ls -la /tmp2
-rw-rw-r-- 1 1001 1001 4 Dec 31 19:31 file1.txt
-rwxrwxr-x 1 1001 1001 6 Dec 31 15:00 file2.txt
-rwxrwxr-x 1 1001 1001 2 Dec 31 15:00 file3.txt

# ls -la /changes
-rw-rw-r-- 1 1001 1001 4 Dec 31 19:31 file1.txt
-rwxrwxr-x 1 1001 1001 2 Dec 31 15:00 file3.txt

md5sum of file1.txt in both directories is equal!

  • Does /tmp1/file1.txt have a different modification time to /tmp2/file1.txt? – MadHatter Feb 3 '16 at 9:14
  • Yes, the /tmp directories are mounted LVM logical volumes. /tmp1 represents the origin LV and /tmp2 a full clone of /tmp1. – J. Pee Feb 3 '16 at 9:52
  • That's interesting, but it makes me unsure that you've answered my question, so I'll repeat it. Are the the files' mtimes different? – MadHatter Feb 3 '16 at 9:54
  • Oh sorry, I thought if I make a copy of something, it should be older than the original one. So yes, the mtime is different. – J. Pee Feb 3 '16 at 10:04
  • The other modification times are equal! – J. Pee Feb 3 '16 at 10:05
1

The man page for rsync notes that

Beginning in version 2.6.4, multiple --compare-dest directories may be provided, which will cause rsync to search the list in the order specified for an exact match. If a match is found that differs only in attributes, a local copy is made and the attributes updated. If a match is not found, a basis file from one of the DIRs will be selected to try to speed up the transfer.

As I read it, that means that if it finds a file which is identical except in terms of attributes in both source and destination, the file isn't transferred from the remote source server; instead, rsync knows the files are identical, so makes the copy in changes/ from local storage but sets the modification time to the newer timestamp of the remote instance.

To confirm this, look at the following output:

[madhatta@risby test]$ ls -la tmp1
total 36
drwxrwxr-x. 2 madhatta madhatta  4096 Feb  3 10:50 .
drwxrwxr-x. 4 madhatta madhatta  4096 Feb  3 11:01 ..
-rw-rw-r--. 1 madhatta madhatta 23150 Feb  3 10:59 file1
-rw-rw-r--. 1 madhatta madhatta   224 Feb  3 10:50 file2
[madhatta@risby test]$ ls -la tmp2
total 36
drwxrwxr-x. 2 madhatta madhatta  4096 Feb  3 10:51 .
drwxrwxr-x. 4 madhatta madhatta  4096 Feb  3 11:01 ..
-rw-rw-r--. 1 madhatta madhatta 23150 Feb  3 11:00 file1
-rw-rw-r--. 1 madhatta madhatta   224 Feb  3 10:50 file2
-rw-rw-r--. 1 madhatta madhatta     0 Feb  3 10:51 file3.txt
[madhatta@risby test]$ rsync -acvu --compare-dest=/home/madhatta/tmp/test/tmp1 tmp2/ changes
sending incremental file list
created directory changes
./
file1
file3.txt

sent 215 bytes  received 71 bytes  572.00 bytes/sec
total size is 23,374  speedup is 81.73

Whilst it reports file1 as having been copied, the amount of data received (70-odd bytes) is in no wise enough to have copied file1 (23kB), compressed or not.

  • Appreciate your explanation! Thanks a lot. Is there a proper way to exclude this local equal copies within the rsync command? Otherwise I will check the hash of all files in /changes and delete same files. – J. Pee Feb 3 '16 at 11:21
  • Not that I know of. Glad you found my answer helpful (and thanks for the acceptance). – MadHatter Feb 3 '16 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.