How would you gracefully handle this snippet to allow for spaces in directories?
If you know the remote system has a xargs
command that supports a -0
option, you can do:
printf '%s\0' "$DEST_PATH/subdir1" "$DEST_PATH/subdir2" |
ssh -i key 10.10.10.10 'xargs -0 mkdir -p --'
That xargs -0 mkdir -p --
piece of shell code would be interpreted the same by all shells (remember sshd
on the remote machine runs the login shell of the remote user to interpret the code given to ssh
which could be anything).
The list of arguments for mkdir
is passed NUL delimited via the remote shell's stdin, inherited by xargs
.
Another advantage with that approach is that you can pass any number of arguments. If needed xargs
will break that list and run several invocations of mkdir -p -- dirn dirn+1...
to bypass the limit of the size of the argument+environment list, even possibly starting the directory creation before the full list has been transferred for very large lists.
See How to execute an arbitrary simple command over ssh without knowing the login shell of the remote user? for other approaches.
rsync
itself by default has the same problem as it passes the name of the remote file/dir to sync from/to as part of that remote shell command line, and the better solution again is to take that remote shell out of the loop by using the -s
/--protect-args
which instead of passing the file/dir names in the shell code, passes it on the rsync
server's stdin.
So:
rsync -s -- "$SOURCE_PATH" "$DEST_HOST:$DEST_PATH/subdir1/"
Quoting the rsync
man page:
If you need to transfer a filename that contains whitespace, you can either specify the
--protect-args
(-s
) option, or you’ll need to escape the whitespace in a way that the remote shell will understand. For instance:rsync -av host:'file\ name\ with\ spaces' /dest
$ rsync --rsync-path='set -x; rsync' /etc/issue localhost:'1 2'
+zsh:1> rsync --server -e.LsfxC . 1 2
$ rsync --rsync-path='set -x; rsync' /etc/issue localhost:'1;uname>&2'
+zsh:1> rsync --server -e.LsfxC . 1
+zsh:1> uname
Linux
See how the file name is interpreted as shell code.
With -s
:
~$ rsync -s --rsync-path='set -x; rsync' /etc/issue localhost:'1;uname>&2'
+zsh:1> rsync --server -se.LsfxC
no trace of the file names on the remote shell command line.
~$ cat '1;uname>&2'
Ubuntu 20.04.1 LTS \n \l
Using double quotes solves the most common issues, but it doesn't solve everything. In your case, double quotes protect against unwanted expansion in the local shell, but then your ssh
command injects the file name directly into a shell snippet without protection, and rsync does the same under the hood for the remote file names passed on its command line¹. That is, if DEST_PATH
is /path/with three spaces
, you're running the command
mkdir -p /path/with three spaces/subdir1 /path/with three spaces/subdir2
on the remote machine. The double quotes around $DEST_PATH
ensure that the shell that's running your script doesn't unduly split the value, but then the spaces are meaningful in the remote shell.
Using nested quotes naively does not solve the problem. For example ssh -i key 10.10.10.10 mkdir -p "'${DEST_PATH}/subdir1'" "'${DEST_PATH}/subdir2'"
would work with file names with spaces, but file names containing '
would not work (and could be a remote command execution vulnerability if an adversary controls the value of DEST_PATH
).
The easy way out is to run sshfs and treat everything as local paths. If you never need to pass file names around in ssh
or rsync
commands, you never need to worry about quoting those file names. However sshfs isn't always convenient to manage, and doesn't lend to efficient updates to large files with rsync.
If the easy way out doesn't work for you, you need to quote the value of DEST_PATH
. That is, you need to change /path/with three spaces
into something like '/path/with three spaces'
or /path/with\ three\ \ spaces
.
set_quoted () {
raw="$1"
quoted=
while case "$raw" in *\'*) true;; *) false;; esac; do
quoted="$quoted'\\''${raw%%\'*}"
raw="${raw#*\'}"
done
quoted="'$quoted$raw'"
}
set_quoted "$DEST_PATH"; quoted_DEST_PATH="$quoted"
ssh -i key 10.10.10.10 mkdir -p "${quoted_DEST_PATH}/subdir1" "${quoted_DEST_PATH}/subdir2"
rsync "${SOURCE_PATH}" "$DEST_HOST:${quoted_DEST_PATH}/subdir1"
¹ Rsync has no problem with arbitrary file names in recursion or in local files. But it passes command line arguments that are a remote file name to a remote shell, which expands them.