How to use Rsync to copy only specific subdirectories (same names in several directories)
I've found the reason. As for me - it wasn't clear that Rsync
works in this way.
So correct command (for company1
directory only) must be:
rsync -avzn --list-only --include 'company1/' --include 'company1/unique_folder1/***' --exclude '*' -e ssh [email protected]:/path/to/old/data/ /path/to/new/data
I.e. we need include each parent company
directory. And of course we cannot write manually all these company
directories in the command line, so we save the list into the file and use it.
Final things we need to do:
1.Generate include file on server 1, so its content will be (I've used ls
and awk
):
+ company1/
+ company1/unique_folder1/***
...
+ companyN/
+ companyN/unique_folder1/***
2.Copy include.txt to server 2 and use such command:
rsync -avzn \
--list-only \
--include-from '/path/to/new/include.txt' \
--exclude '*' \
-e ssh [email protected]:/path/to/old/data/ \
/path/to/new/data
If the first matching pattern excludes a directory, then all its descendants will never be traversed. When you want to include a deep directory e.g. company*/unique_folder1/**
but exclude everything else *
, you need to tell rsync to include all its ancestors too:
rsync -r -v --dry-run \
--include='/' \
--include='/company*/' \
--include='/company*/unique_folder1/' \
--include='/company*/unique_folder1/**' \
--exclude='*'
You can use bash’s brace expansion to save some typing. After brace expansion, the following command is exactly the same as the previous one:
rsync -r -v --dry-run --include=/{,'company*/'{,unique_folder1/{,'**'}}} --exclude='*'
For example, if you only want to sync target/classes/
and target/lib/
to a remote system, do
rsync -vaH --delete --delete-excluded --include='classes/***' --include='lib/***' \
--exclude='*' target/ user@host:/deploy/path/
The important things to watch:
- Don't forget the "
/
" from the end of the pathes, or you will get a copy into subdirectory. - The order of the
--include
,--exclude
counts. - Contrary the other answers, starting with "
/
" an include/exclude parameter is unneeded, they will automatically appended to the source directory (target/
in the example). - To test, what exactly will happen, we can use a
--dry-run
flags, as the other answers say. --delete-excluded
will delete all content in the target directory, except the subdirectories we specifically included. It should be used wisely! On this reason, a--delete
is not enough, it does not deletes the excluded files on the remote side by default (every other, yes), it should be given beside the ordinary--delete
, again.