unexpected behaviour in shell command substitution
Your command substitution will generate a string.
In the case of
$(echo '"hello " "hi and bye"')
this string will be "hello " "hi and bye"
.
The string is then undergoing word splitting (and filename globbing, but it doesn't affect this example). The word splitting happen on every character that is the same as one of the characters in $IFS
(by default, spaces, tabs and newlines).
The words generated by the default value of IFS
would be "hello
, "
, "hi
, and
, and bye"
.
These are then given as separate arguments to your script.
In your second command, the command substitution is
$(echo "hello " "hi and bye")
This generates the string hello hi and bye
and the word splitting would result in the four words hello
, hi
, and
, and bye
.
In your last example, you use the two arguments hello
and hi and bye
directly with your script. These won't undergo word splitting because they are quoted.
TL;DR: the result of echo
is a victim of word splitting on $()
result, and sh test.sh "hello " "hi and bye"
uses a different rule.
Let's take a peak as what's actually happening, when you add set -x
for debugging:
> set -x
> ./main.sh $(echo '"hello " "hi and bye"')
++ echo '"hello " "hi and bye"'
+ ./main.sh '"hello' '"' '"hi' and 'bye"'
"hello
"
"hi
and
bye"
echo
receives "hello " "hi and bye"
as a single argument (yep, including all spaces and single quote) due to single-quotes in the command itself. But command substitution turns that into "hello " "hi and bye"
. To quote bash manual:
If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results.
However, in your case, unquoted command substitution allows exactly for word splitting to occur based on blank character (which is one of the 3 characters set in the $IFS
variable, which is what is used for word splitting). Thus, if you break the resulting string at each space, what do you have?
"hello
, then space, after which we have next token"
- that one character by itself, isolated by spaces from both sides."hi
, again with space on left and rightand
bye"
all in all, you get 5 tokens broken at whitespaces. It's important to realize that word splitting, in this case, occurs exactly a result of command substitution. Later, all of these are treated as individual tokens that have been already parsed. As you see in set -x
output, the shell script is called with exactly those tokens.
As for why sh test.sh "hello " "hi and bye"
behaves differently, that's because the different rule applies. Your command-line is parsed after you enter it, and quoted strings are treated as a single unit, i.e. hello
and hi and bye
You could change IFS
to something other than space (since you want to protect the space in hi and bye
), and use that something to separate the two strings:
$ IFS=:
$ sh test.sh $(echo "hello ":"hi and bye")
hello
hi and bye
The order of operations (those relevant in this example, anyway) is somewhat like this:
- substitutions
- word splitting
- quote removal
Note that quotes created as a result of substitutions aren't special in word splitting or quote removal, so a quote in the output of (1) passes through (2) and (3) like any other character. So:
- In
echo "hello " "hi and bye"
, the shell removes these quotes, soecho
getshello
andhi and bye
, so outputshello hi and bye
, joining the strings with a space. - In
echo '"hello " "hi and bye"'
, the shell removes the outer'
, the echo gets"hello " "hi and bye"
and outputs that. In
sh test.sh $(echo '"hello " "hi and bye"')
, the command substitution is replaced with"hello " "hi and bye"
, but these quotes are a result of a substitution, and so aren't involved in word splitting or quote removal.So the shell splits those into
"hello
,"
,"hi
,and
,bye"
, hence the output you get.- In
sh test.sh $(echo "hello " "hi and bye")
, the command substitution is replaced withhello hi and bye
, which gets split intohello
,hi
andbye
.