Strange "echo" behavior in shell script
To complement chepner's helpful answer:
If output doesn't look what you expect it to look like, it's always worth examining its contents to look for hidden control characters that may change data's appearance on output.
\r
, a CR (carriage return; ASCII value 13) is a notorious example for two reasons:- (As @chepner has stated) it moves the cursor to the beginning of the line mid-string, effectively erasing at least part of what came before it; e.g.:
echo $'abcd\refg'
printsefgd
: the\r
causes everything after to restart printing at the beginning of the line, with only thed
from the string before the\r
surviving, because it happened to be 1 char. longer than the string that came after.
(Note: The$'...'
syntax is a so-called ANSI C-quoted string, which allows use of escape sequences such as\r
in$'...\r...'
to create actual control characters.)
- Files with unexpected
\r
chars. occur frequently when interfacing with the Windows world, where line breaks aren't just\n
chars., but\r\n
sequences, and such files behave strangely in the Unix world.
- (As @chepner has stated) it moves the cursor to the beginning of the line mid-string, effectively erasing at least part of what came before it; e.g.:
- A simple way to examine data is to pipe it to
cat -et
, which highlights control characters as^<char>
sequences:^M
represents a\r
(CR)^I
represents a\t
(tab. char)^[
represents an ESC char.... # see 'man cat'
- The end of a line is represented as
$
- Thus, a file with Windows-style line endings will show
^M$
at the end of the lines output bycat -et
.
cat -et
applied to the above example yields the following, which makes it easy to diagnose the problem:echo $'abcd\refg' | cat -et # -> 'abcd^Mefg$' - note the ^M
dos2unix
is the go-to tool for converting Windows-style line endings (\r\n
) to Unix ones (\r\n
), but this tool doesn't come preinstalled on most Unix-like platforms, and it's easy to use standard POSIX utilities to perform such a conversion:awk 'sub("\r$", "")+1' win.txt > unix.txt
- Note that this POSIX-compliant command doesn't allow you to replace the file in-place, however:
- If you have GNU
sed
, the following would perform the conversion in place:sed -i 's/\r$//' winIn_UnixOut.txt
- Ditto with BSD
sed
(also used on OSX), frombash
,ksh
, orzsh
:sed -i '' $'s/\r$//' winIn_UnixOut.txt
- If you have GNU
Somehow, the value of $string
ends with a carriage return. What you are actually echoing is
echo -ne "\nABCtest.it\r,abc test"
That carriage return causes the cursor to move to the beginning of the current line before printing the text that follows it, overwriting the url. Either remove the carriage return with
string=${string#?}
or fix how string
is set in the first place (I suspect you are reading from a file that uses DOS line endings, in which case you can convert the file with dos2unix
.)