compare 2 directories and copy differences in a 3rd directory

Solution 1:

Use --compare-dest.

From the man page:

--compare-dest=DIR - This option instructs rsync to use DIR on the destination machine as an additional hierarchy to compare destination files against doing transfers (if the files are missing in the destination directory). If a file is found in DIR that is identical to the sender's file, the file will NOT be transferred to the destination directory. This is useful for creating a sparse backup of just files that have changed from an earlier backup.

first check your syntax with --dry-run

rsync -aHxv --progress --dry-run --compare-dest=folder2/ folder1/ folder3/

Then once you're satisfied with the output:

rsync -aHxv --progress  --compare-dest=folder2/ folder1/ folder3/

this link has a good explanation of --compare-dest scope.

Solution 2:

This little script should work:

folder1="firsts_folder/"
folder2="second_folder/"
folder3="target_folder"
for x in `rsync -rcnC --out-format="%n"  $folder1 $folder2`
    do
        if [ -d "$folder1/$x" ]; then
            mkdir -p "$folder3/$x"
        else
            cp -frv $folder1/$x $folder3/$x
        fi
done

But I also think that parsing output of diff -qr would be a better solution because it gives you path and name for each file consistently.

EDIT

This would be one command, your command slightly modified. But it will create folder1 under the folder3. And it does not work with full paths but also the script above does not work with full paths.

rsync -rcnC --out-format="\"%f\"" folder1/ folder2/ | xargs cp --parents -rfvt folder3/

I used \" in rsync to make it work with files and directories containing blanks. Tested on Ubuntu 10.04. Hope it helps.

---

Solution 3:

rsync -rcnC --out-format="%f" . ../folder2/ |xargs cp --parents -rt ../folder3/

That's the simplest form I could come up with, if you're willing to first descend into folder1 before issuing the command.

This is the slower option, but I believe it meets your requirements, and it could be executed using non-relative directory names:

rsync -rcnC --out-format="%n" folder1/ folder2/ |grep -vP "/$" |xargs -I{} rsync -R folder1/./{} folder3/

I left your original rsync command alone, because you said it was giving you the correct file list (and my test files are obviously different than yours, so I couldn't say for certain that I'd be getting the correct file list if I messed with it).

The grep is there to remove directory names from the output of rsync.

The output of grep is then piped to another rsync using xargs. In the param folder1/./{} the extra ./ is used to tell rsync to create the relative directories starting with ./{}, ignoring folder1/. In other words, it avoids creating folder3/folder1/.

I was only able to test this on CentOS 6.4, but AFAIK it should work the same in Ubuntu 12.04.

EDIT: I should note that smithian's answer is undoubtedly better, but I wanted to take a stab at doing it with your original rsync command in tact since it was giving you the desired output.