Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Running ubuntu 12.04, I want to compare 2 directories, say folder1/ and folder2/ and copy any files that are different to folder3/. There are also nested files, so matching subdirectories should be copied as well

Is there a single command that would help me? I can get the full list of changed files running:

rsync -rcnC --out-format="%f" folder1/ folder2/

But rsync doesn't seem to have the ability to "export" these files on a different target directory. Can I pipe the list to cp or some other program, so that the files are copied, while the directories are created as well? For example, I tried

rsync -rcnC --out-format="%f" folder1/ folder2/ | xargs cp -t folder3/

but that wouldn't preserve directories as well, it would simply copy all files inside folder3/

share|improve this question
    
I'd probably use some combination of diff -qr and awk to build the list, then rsync... –  ewwhite May 8 '13 at 13:58
    
how would I use rsync with the list? –  periklis May 8 '13 at 14:15
    
I'd either run a for-loop or use the rsync --include-from=FILE option. –  ewwhite May 8 '13 at 14:22
    
(don't be lazy...) But the rsync solution below is elegant. –  ewwhite May 16 '13 at 22:37

3 Answers 3

up vote 11 down vote accepted
+50

Use --compare-dest.

From the man page:

--compare-dest=DIR - This option instructs rsync to use DIR on the destination machine as an additional hierarchy to compare destination files against doing transfers (if the files are missing in the destination directory). If a file is found in DIR that is identical to the sender's file, the file will NOT be transferred to the destination directory. This is useful for creating a sparse backup of just files that have changed from an earlier backup.

first check your syntax with --dry-run

rsync -aHxv --progress --dry-run --compare-dest=folder2/ folder1/ folder3/

Then once you're satisfied with the output:

rsync -aHxv --progress  --compare-dest=folder2/ folder1/ folder3/

this link has a good explanation of --compare-dest scope.

share|improve this answer
    
Thanks for this, I hadn't understood the man page explanation of this parameter. For anyone interested, I did it with rsync -rcC --compare-dest=folder2/ folder1/ folder3/ –  periklis May 17 '13 at 7:50
    
quick note for OsX users. I have rsync version 3.1.1 protocol version 31 on my local and rsync doesn't understand ~/ paths for --compare-dest param. –  Jevgeni Smirnov Jul 24 at 14:41

This little script should work:

folder1="firsts_folder/"
folder2="second_folder/"
folder3="target_folder"
for x in `rsync -rcnC --out-format="%n"  $folder1 $folder2`
    do
        if [ -d "$folder1/$x" ]; then
            mkdir -p "$folder3/$x"
        else
            cp -frv $folder1/$x $folder3/$x
        fi
done

But I also think that parsing output of diff -qr would be a better solution because it gives you path and name for each file consistently.

EDIT

This would be one command, your command slightly modified. But it will create folder1 under the folder3. And it does not work with full paths but also the script above does not work with full paths.

rsync -rcnC --out-format="\"%f\"" folder1/ folder2/ | xargs cp --parents -rfvt folder3/

I used \" in rsync to make it work with files and directories containing blanks. Tested on Ubuntu 10.04. Hope it helps.

share|improve this answer
    
thanks for this, I had hoped that a single command would exist though –  periklis May 8 '13 at 16:42
rsync -rcnC --out-format="%f" . ../folder2/ |xargs cp --parents -rt ../folder3/

That's the simplest form I could come up with, if you're willing to first descend into folder1 before issuing the command.

This is the slower option, but I believe it meets your requirements, and it could be executed using non-relative directory names:

rsync -rcnC --out-format="%n" folder1/ folder2/ |grep -vP "/$" |xargs -I{} rsync -R folder1/./{} folder3/

I left your original rsync command alone, because you said it was giving you the correct file list (and my test files are obviously different than yours, so I couldn't say for certain that I'd be getting the correct file list if I messed with it).

The grep is there to remove directory names from the output of rsync.

The output of grep is then piped to another rsync using xargs. In the param folder1/./{} the extra ./ is used to tell rsync to create the relative directories starting with ./{}, ignoring folder1/. In other words, it avoids creating folder3/folder1/.

I was only able to test this on CentOS 6.4, but AFAIK it should work the same in Ubuntu 12.04.

EDIT: I should note that smithian's answer is undoubtedly better, but I wanted to take a stab at doing it with your original rsync command in tact since it was giving you the desired output.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.