How do you compare two folders and copy the difference to a third folder?

I have figured out what the problem was in my case:

The files I was comparing had different timestamps. I shouldn't have used the -a argument, I assume because rsync was trying to preserve the timestamps when copying files. The command which worked for me was:

rsync -rvcm --compare-dest=../old/ new/ difference/

I am not sure whether you can do it with any existing linux commands such as rsync or diff. But in my case I had to write my own script using Python, as python has the "filecmp" module for file comparison. I have posted the whole script and usage in my personal site - http://linuxfreelancer.com/

It usage is simple - give it the absolute path of new directory, old directory and difference directory in that order.

#!/usr/bin/env python

import os, sys
import filecmp
import re
from distutils import dir_util
import shutil
holderlist=[]

def compareme(dir1, dir2):
    dircomp=filecmp.dircmp(dir1,dir2)
    only_in_one=dircomp.left_only
    diff_in_one=dircomp.diff_files
    dirpath=os.path.abspath(dir1)
    [holderlist.append(os.path.abspath( os.path.join(dir1,x) )) for x in only_in_one]
    [holderlist.append(os.path.abspath( os.path.join(dir1,x) )) for x in diff_in_one]
    if len(dircomp.common_dirs) > 0:
        for item in dircomp.common_dirs:
            compareme(os.path.abspath(os.path.join(dir1,item)), os.path.abspath(os.path.join(dir2,item)))
        return holderlist

def main():
 if len(sys.argv) > 3:
   dir1=sys.argv[1]
   dir2=sys.argv[2]
   dir3=sys.argv[3]
 else:
   print "Usage: ", sys.argv[0], "currentdir olddir difference"
   sys.exit(1)

 if not dir3.endswith('/'): dir3=dir3+'/'

 source_files=compareme(dir1,dir2)
 dir1=os.path.abspath(dir1)
 dir3=os.path.abspath(dir3)
 destination_files=[]
 new_dirs_create=[]
 for item in source_files:
   destination_files.append(re.sub(dir1, dir3, item) )
 for item in destination_files:
  new_dirs_create.append(os.path.split(item)[0])
 for mydir in set(new_dirs_create):
   if not os.path.exists(mydir): os.makedirs(mydir)
#copy pair
 copy_pair=zip(source_files,destination_files)
 for item in copy_pair:
   if os.path.isfile(item[0]):
    shutil.copyfile(item[0], item[1])

if __name__ == '__main__':
 main()

This might help some readers: In Windows, an older, little freeware program -- Third Dir -- does exactly what's being asked for here. It's no longer available via the developer, Robert Vašíček. But I'm sure it can be found via some repositories online.

Here's the developer's description, which remains on his site:

Third Dir: An unusual directory-synchronizer - the different files are copied to third directory. It is very useful to extract e.g. new or edited photos from a huge directory tree on fixed disk to temporary folder, then add them to archive CD (note - the original files are compared against the CD). Version 1.4, size 23kB. Created 2005-02-12.

History: Version 1.14 - More efficient when many ten of thousands of files are compared.