0

I have the following directory structure:

/tmp1
  \_ file1.txt
  \_ file2.txt

/tmp2
  \_ file1.txt
  \_ file2.txt
  \_ file3.txt
/changes

With rsync's --compare-dest option, I want to sync all files into /changes that are not present in /tmp1 but in /tmp2. Modified files in /tmp2 that differ from /tmp1 should also be synced to /changes.

When I execute the following command:

rsync -acvu --compare-dest=/tmp1/ /tmp2/ changes

file3.txt will be saved in changes. But also file1.txt although the md5 sum of the file in both directories is equal!

Did I miss some parameters for rsync? Thanks in advance!

UPDATE:

I find out that the files that shouldn't be synced have file permission -rw-rw-r-- others have -rwxrwxr-x. I can't find something about this behavior in relation to rsync.

Output of "ls -la" on both directories:

# ls -la /tmp1
-rw-rw-r-- 1 1001 1001 4 Dec 31 15:00 file1.txt
-rwxrwxr-x 1 1001 1001 6 Dec 31 15:00 file2.txt

# ls -la /tmp2
-rw-rw-r-- 1 1001 1001 4 Dec 31 19:31 file1.txt
-rwxrwxr-x 1 1001 1001 6 Dec 31 15:00 file2.txt
-rwxrwxr-x 1 1001 1001 2 Dec 31 15:00 file3.txt

# ls -la /changes
-rw-rw-r-- 1 1001 1001 4 Dec 31 19:31 file1.txt
-rwxrwxr-x 1 1001 1001 2 Dec 31 15:00 file3.txt

md5sum of file1.txt in both directories is equal!

8
  • Does /tmp1/file1.txt have a different modification time to /tmp2/file1.txt?
    – MadHatter
    Feb 3, 2016 at 9:14
  • Yes, the /tmp directories are mounted LVM logical volumes. /tmp1 represents the origin LV and /tmp2 a full clone of /tmp1.
    – J. Pee
    Feb 3, 2016 at 9:52
  • That's interesting, but it makes me unsure that you've answered my question, so I'll repeat it. Are the the files' mtimes different?
    – MadHatter
    Feb 3, 2016 at 9:54
  • Oh sorry, I thought if I make a copy of something, it should be older than the original one. So yes, the mtime is different.
    – J. Pee
    Feb 3, 2016 at 10:04
  • The other modification times are equal!
    – J. Pee
    Feb 3, 2016 at 10:05

1 Answer 1

1

The man page for rsync notes that

Beginning in version 2.6.4, multiple --compare-dest directories may be provided, which will cause rsync to search the list in the order specified for an exact match. If a match is found that differs only in attributes, a local copy is made and the attributes updated. If a match is not found, a basis file from one of the DIRs will be selected to try to speed up the transfer.

As I read it, that means that if it finds a file which is identical except in terms of attributes in both source and destination, the file isn't transferred from the remote source server; instead, rsync knows the files are identical, so makes the copy in changes/ from local storage but sets the modification time to the newer timestamp of the remote instance.

To confirm this, look at the following output:

[madhatta@risby test]$ ls -la tmp1
total 36
drwxrwxr-x. 2 madhatta madhatta  4096 Feb  3 10:50 .
drwxrwxr-x. 4 madhatta madhatta  4096 Feb  3 11:01 ..
-rw-rw-r--. 1 madhatta madhatta 23150 Feb  3 10:59 file1
-rw-rw-r--. 1 madhatta madhatta   224 Feb  3 10:50 file2
[madhatta@risby test]$ ls -la tmp2
total 36
drwxrwxr-x. 2 madhatta madhatta  4096 Feb  3 10:51 .
drwxrwxr-x. 4 madhatta madhatta  4096 Feb  3 11:01 ..
-rw-rw-r--. 1 madhatta madhatta 23150 Feb  3 11:00 file1
-rw-rw-r--. 1 madhatta madhatta   224 Feb  3 10:50 file2
-rw-rw-r--. 1 madhatta madhatta     0 Feb  3 10:51 file3.txt
[madhatta@risby test]$ rsync -acvu --compare-dest=/home/madhatta/tmp/test/tmp1 tmp2/ changes
sending incremental file list
created directory changes
./
file1
file3.txt

sent 215 bytes  received 71 bytes  572.00 bytes/sec
total size is 23,374  speedup is 81.73

Whilst it reports file1 as having been copied, the amount of data received (70-odd bytes) is in no wise enough to have copied file1 (23kB), compressed or not.

2
  • Appreciate your explanation! Thanks a lot. Is there a proper way to exclude this local equal copies within the rsync command? Otherwise I will check the hash of all files in /changes and delete same files.
    – J. Pee
    Feb 3, 2016 at 11:21
  • Not that I know of. Glad you found my answer helpful (and thanks for the acceptance).
    – MadHatter
    Feb 3, 2016 at 11:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .