Size of a folder with du

Since you have copied the files using rsync and then compared the two sets of files using diff, and since diff reports no difference, the two sets of files are identical.

The size difference can then probably be explained by the sizes of the actual directory nodes within the two directory structures. On some filesystems, the directory is not truncated if a file or subdirectory is deleted, leaving a directory node that is slightly larger than what's actually needed.

If you have, at some point, kept many files that were later deleted, this might have left large directory nodes.

Example:

$ mkdir dir
$ ls -ld dir
drwxr-xr-x  2 kk  wheel  512 May 11 17:09 dir

$ touch dir/file-{1..1000}
$ ls -ld dir
drwxr-xr-x  2 kk  wheel  20480 May 11 17:09 dir

$ rm dir/*
$ ls -ld dir
drwxr-xr-x  2 kk  wheel  20480 May 11 17:09 dir
$ du -h .
20.0K   ./dir
42.0K   .
$ ls -R
dir

./dir:

Notice how, even though I deleted the 1000 files I created, the dir directory still uses 20 KB.