Quoting within $(command substitution) in Bash

In order from worst to best:

• DIRNAME="$(dirname $FILE)" will not do what you want if $FILE contains whitespace or globbing characters \[?*.
• DIRNAME=`dirname "$FILE"` is technically correct, but backticks are not recommended for command expansion because of the extra complexity when nesting them.
• DIRNAME=$(dirname "$FILE") is correct, but only because this is an assignment. If you use the command substitution in any other context, such as export DIRNAME=$(dirname "$FILE") or du $(dirname "$FILE"), the lack of quotes will cause trouble if the result of the expansion contain whitespace or globbing characters.
• DIRNAME="$(dirname "$FILE")" is the recommended way. You can replace DIRNAME= with a command and a space without changing anything else, and dirname receives the correct string.

To improve even further:

• DIRNAME="$(dirname -- "$FILE")" works if $FILE starts with a dash.
• DIRNAME="$(dirname -- "$FILE"; printf x)" && DIRNAME="${DIRNAME%?x}" works even if $FILE ends with a newline, since $() chops off newlines at the end of output and dirname outputs a newline after the result. Sheesh dirname, why you gotta be different?

You can nest command expansions as much as you like. With $() you always create a new quoting context, so you can do things like this:

foo "$(bar "$(baz "$(ban "bla")")")"

You do not want to try that with backticks.

You can always show the effects of variable quoting with printf.

Word splitting done on var1:

$ var1="hello     world"
$ printf '[%s]\n' $var1
[hello]
[world]

var1 quoted, so no word splitting:

$ printf '[%s]\n' "$var1"
[hello     world]

Word splitting on var1 inside $(), equivalent to echo "hello" "world":

$ var2=$(echo $var1)
$ printf '[%s]\n' "$var2"
[hello world]

No word splitting on var1, no problem with not quoting the $():

$ var2=$(echo "$var1")
$ printf '[%s]\n' "$var2"
[hello     world]

Word splitting on var1 again:

$ var2="$(echo $var1)"
$ printf '[%s]\n' "$var2"
[hello world]

Quoting both, easiest way to be sure.

$ var2="$(echo "$var1")"
$ printf '[%s]\n' "$var2"
[hello     world]

Globbing problem

Not quoting a variable can also lead to glob expansion of its contents:

$ mkdir test; cd test; touch file1 file2
$ var="*"
$ printf '[%s]\n' $var
[file1]
[file2]
$ printf '[%s]\n' "$var"
[*]

Note this happens after the variable is expanded only. It is not necessary to quote a glob during assignment:

$ var=*
$ printf '[%s]\n' $var
[file1]
[file2]
$ printf '[%s]\n' "$var"
[*]

Use set -f to disable this behaviour:

$ set -f
$ var=*
$ printf '[%s]\n' $var
[*]

And set +f to re-enable it:

$ set +f
$ printf '[%s]\n' $var
[file1]
[file2]

Addition to the accepted answer:

While I generally agree with @l0b0's answer here, I suspect the placement of bare backticks in the "worst to best" list is at least partly a result of the assumption that $(...) is available everywhere. I realize that the question specifies Bash, but there are plenty of times when Bash turns out to mean /bin/sh, which may not always actually be the full Bourne Again shell.

In particular, the plain Bourne shell won't know what to do with $(...), so scripts which claim to be compatible with it (e.g., via a #!/bin/sh shebang line) will likely misbehave if they are actually run by the "real" /bin/sh – this is of special interest when, say, producing init scripts, or packaging pre- and post-scripts, and can land one in a surprising place during installation of a base system.

If any of that sounds like something you're planning to do with this variable, nesting is probably less of a concern than having the script actually, predictably run. When it's a simple enough case and portability is a concern, even if I expect the script to usually run on systems where /bin/sh is Bash, I often tend to use backticks for this reason, with multiple assignments instead of nesting.

Having said all that, the ubiquity of shells which implement $(...) (Bash, Dash, et al.), leaves us in a good spot to stick with the prettier, easier-to-nest, and more recently preferred POSIX syntax in most cases, for all the reasons @l0b0 mentions.

Aside: this has shown up occasionally on StackOverflow, too –

• Command substitution: backticks or dollar sign / paren enclosed? [duplicate] (Feb 2012)
• Shell Programming: What's the difference between $(command) and `command` (Jan 2011)