compare 2 directories and copy differences in a 3rd directory

Question

Running ubuntu 12.04, I want to compare 2 directories, say folder1/ and folder2/ and copy any files that are different to folder3/. There are also nested files, so matching subdirectories should be copied as well

Is there a single command that would help me? I can get the full list of changed files running:

rsync -rcnC --out-format="%f" folder1/ folder2/

But rsync doesn't seem to have the ability to "export" these files on a different target directory. Can I pipe the list to cp or some other program, so that the files are copied, while the directories are created as well? For example, I tried

rsync -rcnC --out-format="%f" folder1/ folder2/ | xargs cp -t folder3/

but that wouldn't preserve directories as well, it would simply copy all files inside folder3/

Answer 1

Use --compare-dest.

From the man page:

--compare-dest=DIR - This option instructs rsync to use DIR on the destination machine as an additional hierarchy to compare destination files against doing transfers (if the files are missing in the destination directory). If a file is found in DIR that is identical to the sender's file, the file will NOT be transferred to the destination directory. This is useful for creating a sparse backup of just files that have changed from an earlier backup.

first check your syntax with --dry-run

rsync -aHxv --progress --dry-run --compare-dest=folder2/ folder1/ folder3/

Then once you're satisfied with the output:

rsync -aHxv --progress  --compare-dest=folder2/ folder1/ folder3/

this link has a good explanation of --compare-dest scope.

Answer 2

This little script should work:

folder1="firsts_folder/"
folder2="second_folder/"
folder3="target_folder"
for x in `rsync -rcnC --out-format="%n"  $folder1 $folder2`
    do
        if [ -d "$folder1/$x" ]; then
            mkdir -p "$folder3/$x"
        else
            cp -frv $folder1/$x $folder3/$x
        fi
done

But I also think that parsing output of diff -qr would be a better solution because it gives you path and name for each file consistently.

EDIT

This would be one command, your command slightly modified. But it will create folder1 under the folder3. And it does not work with full paths but also the script above does not work with full paths.

rsync -rcnC --out-format="\"%f\"" folder1/ folder2/ | xargs cp --parents -rfvt folder3/

I used \" in rsync to make it work with files and directories containing blanks. Tested on Ubuntu 10.04. Hope it helps.

Answer 3

rsync -rcnC --out-format="%f" . ../folder2/ |xargs cp --parents -rt ../folder3/

That's the simplest form I could come up with, if you're willing to first descend into folder1 before issuing the command.

This is the slower option, but I believe it meets your requirements, and it could be executed using non-relative directory names:

rsync -rcnC --out-format="%n" folder1/ folder2/ |grep -vP "/$" |xargs -I{} rsync -R folder1/./{} folder3/

I left your original rsync command alone, because you said it was giving you the correct file list (and my test files are obviously different than yours, so I couldn't say for certain that I'd be getting the correct file list if I messed with it).

The grep is there to remove directory names from the output of rsync.

The output of grep is then piped to another rsync using xargs. In the param folder1/./{} the extra ./ is used to tell rsync to create the relative directories starting with ./{}, ignoring folder1/. In other words, it avoids creating folder3/folder1/.

I was only able to test this on CentOS 6.4, but AFAIK it should work the same in Ubuntu 12.04.

EDIT: I should note that smithian's answer is undoubtedly better, but I wanted to take a stab at doing it with your original rsync command in tact since it was giving you the desired output.