echo vs <<<, or Useless Use of echo in Bash Award?

First, let's concentrate on performance. I ran benchmarks for a slightly different program on an otherwise mostly idle x86_64 processor running Debian squeeze.

herestring.bash, using a herestring to pass a line of input:

#! /bin/bash
i=0
while [ $i -lt $1 ]; do
  tr a-z A-Z <<<'hello world'
  i=$((i+1))
done >/dev/null

heredoc.bash, using a heredoc to pass a line of input:

#! /bin/bash
i=0
while [ $i -lt $1 ]; do
  tr a-z A-Z <<'EOF'
hello world
EOF
  i=$((i+1))
done >/dev/null

echo.bash, using echo and a pipe to pass a line of input:

#! /bin/bash
i=0
while [ $i -lt $1 ]; do
  echo 'hello world' | tr a-z A-Z
  i=$((i+1))
done >/dev/null

For comparison, I also timed the scripts under ATT ksh93 and under dash (except for herestring.bash, because dash doesn't have herestrings).

Here are median-of-three times:

$ time bash ./herestring.bash 10000
./herestring.bash 10000  0.32s user 0.79s system 15% cpu 7.088 total
$ time ksh ./herestring.bash 10000
ksh ./herestring.bash 10000  0.54s user 0.41s system 17% cpu 5.277 total
$ time bash ./heredoc.bash 10000
./heredoc.bash 10000  0.35s user 0.75s system 17% cpu 6.406 total
$ time ksh ./heredoc.bash 10000  
ksh ./heredoc.sh 10000  0.54s user 0.44s system 19% cpu 4.925 total
$ time sh ./heredoc.bash 10000  
./heredoc.sh 10000  0.08s user 0.58s system 12% cpu 5.313 total
$ time bash ./echo.bash 10000
./echo.bash 10000  0.36s user 1.40s system 20% cpu 8.641 total
$ time ksh ./echo.bash 10000
ksh ./echo.sh 10000  0.47s user 1.51s system 28% cpu 6.918 total
$ time sh ./echo.sh 10000
./echo.sh 10000  0.07s user 1.00s system 16% cpu 6.463 total

Conclusions:

  • A heredoc is faster than a herestring.
  • echo and a pipe is noticeably, but not dramatically faster. (Keep in mind that this is a toy program: in a real program, most of the processing time would be in whatever the tr call stands for here.)
  • If you want speed, ditch bash and call dash or even better ksh instead. Bash's features don't make up for its relative slowness, but ksh has both features and speed.

Beyond performance, there's also clarity and portability. <<< is a ksh93/bash/zsh extension which is less well-known than echo … | or <<. It doesn't work in ksh88/pdksh or in POSIX sh.

The only place where <<< is arguably significantly clearer is inside a heredoc:

foo=$(tr a-z A-Z <<<'hello world')

vs

foo=$(tr a-z A-Z <<'EOF'
hello world
EOF
)

(Most shells can't cope with closing the parenthesis at the end of the line containing <<EOF.)


Another reason to use heredocs (if you didn't have enough) is that echo can fail if the stream isn't consumed. Consider having bash' pipefail option:

set -o pipefail
foo=yawn
echo $foo | /bin/true ; echo $?  # returns 0

/bin/true doesn't consume its standard input, but echo yawn completes nonetheless. However, if echo is asked to print a lot of data, it will not complete until after true has completed:

foo=$(cat /etc/passwd)
# foo now has a fair amount of data

echo $foo | /bin/true ; echo $?  # returns 0 sometimes 141
echo $foo$foo$foo$foo | /bin/true ; echo $?  # returns mostly 141

141 is SIGPIPE (128 + 13) (128 being added because bash does so according to bash(1):

When a command terminates on a fatal signal N, bash uses the value of 128+N as the exit status.

Heredocs don't have this problem:

/bin/true <<< $foo$foo$foo$foo ; echo $?  # returns 0 always

One reason you might want to use echo is to exhert some control the newline character that is added to the end of heredocs and herestrings:

Three characters foo has length 3:

$ echo -n foo | wc -c
3

However, a threecharacter herestring is four characters:

$ wc -c <<< foo
4

A three-character heredoc too:

$ wc -c << EOF
foo
EOF
4

The fourth character is a newline 0x0a character.

Somehow this magically fits in with the way bash removes these newline characters when grabbing output from a sub-shell:

Here is a command that returns four characters: foo and \n. The '\n' is added by echo, it always adds a newline character unless you specify the -n option:

$ echo foo
foo
$ echo foo | wc -c
4

However, by assigning this to a variable, the trailing newline added by echo is removed:

$ foo=$(echo foo)
$ echo "$foo" # here, echo adds a newline too.
foo

So if you mix files and variables and use them in calculations (e.g. , you can't use heredocs or herestrings, since they will add a newline.

foo=abc
echo -n 'abc' > something.txt
if [ $(wc -c <<< "$foo") -eq $(wc -c < something.txt) ] ; then
  echo "yeah, they're the same!"
else
  echo "foo and bar have different lengths (except, maybe not)"
fi

If you change the if statement to read

if [ $(echo -n "$foo" | wc -c) -eq $(wc -c < something.txt) ] ; then

then the test passes.