Using Rsync include and exclude options to include directory and file by pattern
Here's my "teach a person to fish" answer:
Rsync's syntax is definitely non-intuitive, but it is worth understanding.
- First, use
-vvv
to see the debug info for rsync.
$ rsync -nr -vvv --include="**/file_11*.jpg" --exclude="*" /Storage/uploads/ /website/uploads/
[sender] hiding directory 1280000000 because of pattern *
[sender] hiding directory 1260000000 because of pattern *
[sender] hiding directory 1270000000 because of pattern *
The key concept here is that rsync applies the include/exclude patterns for each directory recursively. As soon as the first include/exclude is matched, the processing stops.
The first directory it evaluates is /Storage/uploads
. Storage/uploads
has 1280000000/, 1260000000/, 1270000000/
dirs/files. None of them match file_11*.jpg
to include. All of them match *
to exclude. So they are excluded, and rsync ends.
- The solution is to include all dirs (
*/
) first. Then the first dir component will be1260000000/, 1270000000/, 1280000000/
since they match*/
. The next dir component will be1260000000/
. In1260000000/
,file_11_00.jpg
matches--include="file_11*.jpg"
, so it is included. And so forth.
$ rsync -nrv --include='*/' --include="file_11*.jpg" --exclude="*" /Storage/uploads/ /website/uploads/
./
1260000000/
1260000000/file_11_00.jpg
1260000000/file_11_01.jpg
1270000000/
1270000000/file_11_00.jpg
1270000000/file_11_01.jpg
1280000000/
1280000000/file_11_00.jpg
1280000000/file_11_01.jpg
https://download.samba.org/pub/rsync/rsync.1
The problem is that --exclude="*"
says to exclude (for example) the 1260000000/
directory, so rsync
never examines the contents of that directory, so never notices that the directory contains files that would have been matched by your --include
.
I think the closest thing to what you want is this:
rsync -nrv --include="*/" --include="file_11*.jpg" --exclude="*" /Storage/uploads/ /website/uploads/
(which will include all directories, and all files matching file_11*.jpg
, but no other files), or maybe this:
rsync -nrv --include="/[0-9][0-9][0-9]0000000/" --include="file_11*.jpg" --exclude="*" /Storage/uploads/ /website/uploads/
(same concept, but much pickier about the directories it will include).
rsync
include exclude pattern examples:
"*" means everything
"dir1" transfers empty directory [dir1]
"dir*" transfers empty directories like: "dir1", "dir2", "dir3", etc...
"file*" transfers files whose names start with [file]
"dir**" transfers every path that starts with [dir] like "dir1/file.txt", "dir2/bar/ffaa.html", etc...
"dir***" same as above
"dir1/*" does nothing
"dir1/**" does nothing
"dir1/***" transfers [dir1] directory and all its contents like "dir1/file.txt", "dir1/fooo.sh", "dir1/fold/baar.py", etc...
And final note is that simply dont rely on asterisks that are used in the beginning for evaluating paths; like "**dir"
(its ok to use them for single folders or files but not paths) and note that more than two asterisks dont work for file names.