A variable modified inside a while loop is not remembered
You are the 742342nd user to ask this bash FAQ. The answer also describes the general case of variables set in subshells created by pipes:
E4) If I pipe the output of a command into
read variable
, why doesn't the output show up in$variable
when the read command finishes?This has to do with the parent-child relationship between Unix processes. It affects all commands run in pipelines, not just simple calls to
read
. For example, piping a command's output into awhile
loop that repeatedly callsread
will result in the same behavior.Each element of a pipeline, even a builtin or shell function, runs in a separate process, a child of the shell running the pipeline. A subprocess cannot affect its parent's environment. When the
read
command sets the variable to the input, that variable is set only in the subshell, not the parent shell. When the subshell exits, the value of the variable is lost.Many pipelines that end with
read variable
can be converted into command substitutions, which will capture the output of a specified command. The output can then be assigned to a variable:grep ^gnu /usr/lib/news/active | wc -l | read ngroup
can be converted into
ngroup=$(grep ^gnu /usr/lib/news/active | wc -l)
This does not, unfortunately, work to split the text among multiple variables, as read does when given multiple variable arguments. If you need to do this, you can either use the command substitution above to read the output into a variable and chop up the variable using the bash pattern removal expansion operators or use some variant of the following approach.
Say /usr/local/bin/ipaddr is the following shell script:
#! /bin/sh host `hostname` | awk '/address/ {print $NF}'
Instead of using
/usr/local/bin/ipaddr | read A B C D
to break the local machine's IP address into separate octets, use
OIFS="$IFS" IFS=. set -- $(/usr/local/bin/ipaddr) IFS="$OIFS" A="$1" B="$2" C="$3" D="$4"
Beware, however, that this will change the shell's positional parameters. If you need them, you should save them before doing this.
This is the general approach -- in most cases you will not need to set $IFS to a different value.
Some other user-supplied alternatives include:
read A B C D << HERE $(IFS=.; echo $(/usr/local/bin/ipaddr)) HERE
and, where process substitution is available,
read A B C D < <(IFS=.; echo $(/usr/local/bin/ipaddr))
UPDATED#2
Explanation is in Blue Moons's answer.
Alternative solutions:
Eliminate echo
while read line; do
...
done <<EOT
first line
second line
third line
EOT
Add the echo inside the here-is-the-document
while read line; do
...
done <<EOT
$(echo -e $lines)
EOT
Run echo
in background:
coproc echo -e $lines
while read -u ${COPROC[0]} line; do
...
done
Redirect to a file handle explicitly (Mind the space in < <
!):
exec 3< <(echo -e $lines)
while read -u 3 line; do
...
done
Or just redirect to the stdin
:
while read line; do
...
done < <(echo -e $lines)
And one for chepner
(eliminating echo
):
arr=("first line" "second line" "third line");
for((i=0;i<${#arr[*]};++i)) { line=${arr[i]};
...
}
Variable $lines
can be converted to an array without starting a new sub-shell. The characters \
and n
has to be converted to some character (e.g. a real new line character) and use the IFS (Internal Field Separator) variable to split the string into array elements. This can be done like:
lines="first line\nsecond line\nthird line"
echo "$lines"
OIFS="$IFS"
IFS=$'\n' arr=(${lines//\\n/$'\n'}) # Conversion
IFS="$OIFS"
echo "${arr[@]}", Length: ${#arr[*]}
set|grep ^arr
Result is
first line\nsecond line\nthird line
first line second line third line, Length: 3
arr=([0]="first line" [1]="second line" [2]="third line")
Hmmm... I would almost swear that this worked for the original Bourne shell, but don't have access to a running copy just now to check.
There is, however, a very trivial workaround to the problem.
Change the first line of the script from:
#!/bin/bash
to
#!/bin/ksh
Et voila! A read at the end of a pipeline works just fine, assuming you have the Korn shell installed.
echo -e $lines | while read line
...
done
The while
loop is executed in a subshell. So any changes you do to the variable will not be available once the subshell exits.
Instead you can use a here string to re-write the while loop to be in the main shell process; only echo -e $lines
will run in a subshell:
while read line
do
if [[ "$line" == "second line" ]]
then
foo=2
echo "Variable \$foo updated to $foo inside if inside while loop"
fi
echo "Value of \$foo in while loop body: $foo"
done <<< "$(echo -e "$lines")"
You can get rid of the rather ugly echo
in the here-string above by expanding the backslash sequences immediately when assigning lines
. The $'...'
form of quoting can be used there:
lines=$'first line\nsecond line\nthird line'
while read line; do
...
done <<< "$lines"