File content into unix variable with newlines

The assignment does not remove the newline characters, it's actually the echo doing this. You need simply put quotes around the string to maintain those newlines:

echo "$testvar"

This will give the result you want. See the following transcript for a demo:

pax> cat num1.txt ; x=$(cat num1.txt)
line 1
line 2

pax> echo $x ; echo '===' ; echo "$x"
line 1 line 2
===
line 1
line 2

The reason why newlines are replaced with spaces is not entirely to do with the echo command, rather it's a combination of things.

When given a command line, bash splits it into words according to the documentation for the IFS variable:

IFS: The Internal Field Separator that is used for word splitting after expansion ... the default value is <space><tab><newline>.

That specifies that, by default, any of those three characters can be used to split your command into individual words. After that, the word separators are gone, all you have left is a list of words.

Combine that with the echo documentation (a bash internal command), and you'll see why the spaces are output:

echo [-neE] [arg ...]: Output the args, separated by spaces, followed by a newline.

When you use echo "$x", it forces the entire x variable to be a single word according to bash, hence it's not split. You can see that with:

pax> function count {
...>    echo $#
...> }
pax> count 1 2 3
3
pax> count a b c d
4
pax> count $x
4
pax> count "$x"
1

Here, the count function simply prints out the number of arguments given. The 1 2 3 and a b c d variants show it in action.

Then we try it with the two variations on the x variable. The one without quotes shows that there are four words, "test", "1", "test" and "2". Adding the quotes makes it one single word "test 1\ntest 2".

This is due to IFS (Internal Field Separator) variable which contains newline.

$ cat xx1
1
2

$ A=`cat xx1`
$ echo $A
1 2

$ echo "|$IFS|"
|       
|

A workaround is to reset IFS to not contain the newline, temporarily:

$ IFSBAK=$IFS
$ IFS=" "
$ A=`cat xx1` # Can use $() as well
$ echo $A
1
2
$ IFS=$IFSBAK

To REVERT this horrible change for IFS:

IFS=$IFSBAK

Your variable is set correctly by testvar=$(cat test.txt). To display this variable which consist new line characters, simply add double quotes, e.g.

echo "$testvar"

Here is the full example:

$ printf "test1\ntest2" > test.txt
$ testvar=$(<test.txt)
$ grep testvar <(set)
testvar=$'test1\ntest2'
$ echo "$testvar"
text1
text2
$ printf "%b" "$testvar"
text1
text2

Bash -ge 4 has the mapfile builtin to read lines from the standard input into an array variable.

help mapfile 

mapfile < file.txt lines
printf "%s" "${lines[@]}"

mapfile -t < file.txt lines    # strip trailing newlines
printf "%s\n" "${lines[@]}"

File content into unix variable with newlines

Tags:

Unix

Bash

Related

Recent Posts