Transferring millions of files from one server to another
As you say, use rsync
:
rsync -azP /var/www/html/txt/ username@ip-address:/var/www/html/txt
The options are:
-a : enables archive mode, which preserves symbolic links and works recursively
-z : compress the data transfer to minimise network usage
-P : to display a progress bar and enables you to resume partial transfers
As @aim says in his answer, make sure you have a trailing /
on the source directory (on both is fine too).
More info from the man page
Just use rsync over ssh!
rsync -av username@ip:/var/www/html/txt /var/www/html/
From the man page:
-a, --archive : This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and want to preserve almost everything (with -H being a notable omission). The only exception to the above equivalence is when --files-from is specified, in which case -r is not implied.
-v, --verbose : This option increases the amount of information you are given during the transfer. By default, rsync works silently. A single -v will give you information about what files are being transferred and a brief summary at the end. Two -v options will give you information on what files are being skipped and slightly more information at the end. More than two -v options should only be used if you are debugging rsync.
Note how did I use slashes at the end of the folders - it's important.