Forcing bash to expand variables in a string loaded from a file

Many of the answers using eval and echo kind of work, but break on various things, such as multiple lines, attempting to escaping shell meta-characters, escapes inside the template not intended to be expanded by bash, etc.

I had the same issue, and wrote this shell function, which as far as I can tell, handles everything correctly. This will still strip only trailing newlines from the template, because of bash's command substitution rules, but I've never found that to be an issue as long as everything else remains intact.

apply_shell_expansion() {
    declare file="$1"
    declare data=$(< "$file")
    declare delimiter="__apply_shell_expansion_delimiter__"
    declare command="cat <<$delimiter"$'\n'"$data"$'\n'"$delimiter"
    eval "$command"
}

For example, you can use it like this with a parameters.cfg which is really a shell script that just sets variables, and a template.txt which is a template that uses those variables:

. parameters.cfg
printf "%s\n" "$(apply_shell_expansion template.txt)" > result.txt

In practice, I use this as a sort of lightweight template system.

I stumbled on what I think is THE answer to this question: the envsubst command:

echo "hello \$FOO world" > source.txt
export FOO=42
envsubst < source.txt

This outputs: hello 42 world

If you would like to continue work on the data in a file destination.txt, push this back to a file like this:

envsubst < source.txt > destination.txt

In case it's not already available in your distro, it's in the GNU package gettext.

@Rockallite

• I wrote a little wrapper script to take care of the '$' problem.

(BTW, there is a "feature" of envsubst, explained at https://unix.stackexchange.com/a/294400/7088 for expanding only some of the variables in the input, but I agree that escaping the exceptions is much more convenient.)

Here's my script:

#! /bin/bash
      ## -*-Shell-Script-*-
CmdName=${0##*/}
Usage="usage: $CmdName runs envsubst, but allows '\$' to  keep variables from
    being expanded.
  With option   -sl   '\$' keeps the back-slash.
  Default is to replace  '\$' with '$'
"

if [[ $1 = -h ]]  ;then echo -e >&2  "$Usage" ; exit 1 ;fi
if [[ $1 = -sl ]] ;then  sl='\'  ; shift ;fi

sed 's/\\\$/\${EnVsUbDolR}/g' |  EnVsUbDolR=$sl\$  envsubst  "$@"