How to make bash glob a string variable?
You can force another round of evaluation with eval
, but that's not actually necessary. (And eval
starts having serious problems the moment your file names contain special characters like $
.) The problem isn't with globbing, but with the tilde expansion.
Globbing happens after variable expansion, if the variable is unquoted, as here(*):
$ x="/tm*" ; echo $x
/tmp
So, in the same vein, this is similar to what you did, and works:
$ mkdir -p ~/public/foo/ ; touch ~/public/foo/x.launch
$ i="$HOME/public/*"; j="*.launch"; k="$i/$j"
$ echo $k
/home/foo/public/foo/x.launch
But with the tilde it doesn't:
$ i="~/public/*"; j="*.launch"; k="$i/$j"
$ echo $k
~/public/*/*.launch
This is clearly documented for Bash:
The order of expansions is: brace expansion; tilde expansion, parameter and variable expansion, ...
Tilde expansion happens before variable expansion so tildes inside variables are not expanded. The easy workaround is
to use $HOME
or the full path instead.
(* expanding globs from variables is usually not what you want)
Another thing:
When you loop over the patterns, as here:
exclude="foo *bar"
for j in $exclude ; do
...
note that as $exclude
is unquoted, it's both split, and also globbed at this point. So if the current directory contains something matching the pattern, it's expanded to that:
$ i="$HOME/public/foo"
$ exclude="*.launch"
$ touch $i/real.launch
$ for j in $exclude ; do # split and glob, no match
echo "$i"/$j ; done
/home/foo/public/foo/real.launch
$ touch ./hello.launch
$ for j in $exclude ; do # split and glob, matches in current dir!
echo "$i"/$j ; done
/home/foo/public/foo/hello.launch # not the expected result
To work around this, use an array variable instead of a splitted string:
$ exclude=("*.launch")
$ exclude+=("something else")
$ for j in "${exclude[@]}" ; do echo "$i"/$j ; done
/home/foo/public/foo/real.launch
/home/foo/public/foo/something else
As an added bonus, array entries can also contain whitespace without issues with splitting.
Something similar could be done with find -path
, if you don't mind what directory level the targeted files should be. E.g. to find any path ending in /e2e/*.js
:
$ dirs="$HOME/public $HOME/private"
$ pattern="*/e2e/*.js"
$ find $dirs -path "$pattern"
/home/foo/public/one/two/three/e2e/asdf.js
We have to use $HOME
instead of ~
for the same reason as before, and $dirs
needs to be unquoted on the find
command line so it gets split, but $pattern
should be quoted so it isn't accidentally expanded by the shell.
(I think you could play with -maxdepth
on GNU find to limit how deep the search goes, if you care, but that's a bit of a different issue.)
You can save it as an array instead of a string to use later in many cases and let the globbing happen when you define it. In your case, for example:
k=(~/code/public/*/*.launch)
for i in "${k[@]}"; do
or in the later example, you'll need to eval
some of the strings
dirs=(~/code/private/* ~/code/public/*)
for i in "${dirs[@]}"; do
for j in $exclude; do
eval "for k in $i/$j; do tmutil addexclusion \"\$k\"; done"
done
done
@ilkkachu answer solved the main globbing issue. Full credit to him.
V1
However, due to exclude
containing entries both with and without wildcard(*), and also they may not exist in all, extra checking is needed after the globbing of $i/$j
. I am sharing my findings here.
#!/bin/bash
exclude="
*.launch
.DS_Store
.classpath
.sass-cache
.settings
Thumbs.db
bower_components
build
connect.lock
coverage
dist
e2e/*.js
e2e/*.map
libpeerconnection.log
node_modules
npm-debug.log
testem.log
tmp
typings
"
dirs="
$HOME/code/private/*
$HOME/code/public/*
"
# loop $dirs
for i in $dirs; do
for j in $exclude ; do
for k in $i/$j; do
echo -e "$k"
if [ -f $k ] || [ -d $k ] ; then
# Only execute command if dir/file exist
echo -e "\t^^^ Above file/dir exist! ^^^"
fi
done
done
done
Output Explaination
Following is the partial output to explain the situation.
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/a.launch
^^^ Above file/dir exist! ^^^
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/b.launch
^^^ Above file/dir exist! ^^^
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/.DS_Store
^^^ Above file/dir exist! ^^^
The above are self explanatory.
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/.classpath
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/.sass-cache
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/.settings
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/Thumbs.db
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/bower_components
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/build
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/connect.lock
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/coverage
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/dist
The above show up because the exclude entry($j
) has no wildcard, $i/$j
become a plain string concatenation. However the file/dir does not exist.
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/e2e/*.js
/Volumes/HD2/JS/code/public/simple-api-example-ng2-express/e2e/*.map
The above show up as exclude entry($j
) contain wildcard but has no file/directory match, the globbing of $i/$j
just return the original string.
V2
V2 use single quote, eval
and shopt -s nullglob
to get clean result. No file/dir final checking require.
#!/bin/bash
exclude='
*.launch
.sass-cache
Thumbs.db
bower_components
build
connect.lock
coverage
dist
e2e/*.js
e2e/*.map
libpeerconnection.log
node_modules
npm-debug.log
testem.log
tmp
typings
'
dirs='
$HOME/code/private/*
$HOME/code/public/*
'
for i in $dirs; do
for j in $exclude ; do
shopt -s nullglob
eval "k=$i/$j"
for l in $k; do
echo $l
done
shopt -u nullglob
done
done