Properly escaping output from pipe in xargs
The POSIX specification does give you an example for that:
ls | sed -e 's/"/"\\""/g' -e 's/.*/"&"/' | xargs -E '' printf '<%s>\n'
(with filenames being arbitrary sequences of bytes (other than /
and NULL) and sed
/xargs
expecting text, you'd also need to fix the locale to C (where all non-NUL bytes would make valid characters) to make that reliable (except for xargs
implementations that have a very low limit on the maximum length of an argument))
The -E ''
is needed for some xargs
implementations that without it, would understand a _
argument to signify the end of input (where echo a _ b | xargs
outputs a
only for instance).
With GNU xargs
, you can use:
ls | xargs -d '\n' printf '<%s>\n'
GNU xargs
has also a -0
that has been copied by a few other implementations, so:
ls | tr '\n' '\0' | xargs -0 printf '<%s>\n'
is slightly more portable.
All of those assume the file names don't contain newline characters. If there may be filenames with newline characters, the output of ls
is simply not post-processable. If you get:
a
b
That can be either two a
and b
files or a file called a<newline>b
, there's no way to tell.
GNU ls
has a --quoting-style=shell-always
which makes its output unambiguous and could be post-processable, but the quoting is not compatible with the quoting expected by xargs
. xargs
recognise "..."
, \x
and '...'
forms of quoting. But both "..."
and '...'
are strong quotes and can't contain newline characters (only \
can escape newline characters for xargs
), so that's not compatible with sh quoting where only '...'
are strong quotes (and can contain newline characters) but \<newline>
is a line-continuation (is removed) instead of an escaped newline.
You can use the shell to parse that output and then output it in a format expected by xargs
:
eval "files=($(ls --quoting-style=shell-always))"
[ "${#files[#]}" -eq 0 ] || printf '%s\0' "${files[@]}" |
xargs -0 printf '<%s>\n'
For xargs
to understand the -0
null-delimited input option, the sending party must also apply the null delimiter to the data that they are sending over.
Else there's no synchronization between the two.
One option is the GNU find
command which can place such delimiters:
find . -maxdepth 1 ! -name . -print0 | xargs -0 ls -ld
As you said, xargs
doesn't like unmatched double quotes unless you use -0
but -0
only makes sense if you feed it null-terminated data. So, this fails:
$ echo * | xargs
xargs: unmatched double quote; by default quotes are special to xargs unless you use the -0 option
name-with-backslash -name-with-dash-prefix
But this works:
$ printf '%s\0' -- * | xargs -0
-- name-with-backslash\ -name-with-dash-prefix name-with-double-quote" name-with-single-quote' name with space safe-name
In any case, your basic approach is not really the best way to do this. Instead of fiddling about with xargs
and ls
and whatnot, just use shell globs instead:
$ for f in *; do ls -l -- "$f"; done
-rw-r--r-- 1 terdon terdon 4142 Aug 11 16:03 a
-rw-r--r-- 1 terdon terdon 0 Aug 11 15:34 'name-with-backslash\'
-rw-r--r-- 1 terdon terdon 0 Aug 11 15:34 -name-with-dash-prefix
-rw-r--r-- 1 terdon terdon 0 Aug 11 15:34 'name-with-double-quote"'
-rw-r--r-- 1 terdon terdon 0 Aug 11 15:34 "name-with-single-quote'"
-rw-r--r-- 1 terdon terdon 0 Aug 11 15:34 'name with space'
-rw-r--r-- 1 terdon terdon 0 Aug 11 15:34 safe-name