When is xargs needed?

The difference is in what data the target program is accepting.

If you just use a pipe, it receives data on STDIN (the standard input stream) as a raw pile of data that it can sort through one line at a time. However some programs don't accept their commands on standard in, they expect it to be spelled out in the arguments to the command. For example touch takes a file name as a parameter on the command line like so: touch file1.txt.

If you have a program that outputs filenames on standard out and want to use them as arguments to touch, you have to use xargs which reads the STDIN stream data and converts each line into space separated arguments to the command.

These two things are equivalent:

# touch file1.txt
# echo file1.txt | xargs touch

Don't use xargs unless you know exactly what it's doing and why it's needed. It's quite often the case that there is a better way to do the job than using xargs to force the conversion. The conversion process is also fraught with potential pitfalls like escaping and word expansion etc.

To expand on the answers already provided, xargs can do one cool thing that is becoming increasingly important in today's multicore and distributed computing landscape: it can parallel process jobs.

For example:

$ find . -type f -name '*.wav' -print0 |xargs -0 -P 3 -n 1 flac -V8

will encode *.wav => *.flac, using three processes at once (-P 3).

xargs is particularly useful when you have a list of filepaths on stdin and want to do something with them. For example:

$ git ls-files "*.tex" | xargs -n 1 sed -i "s/color/colour/g"

Let's examine this step by step:

$ git ls-files "*.tex"
tex/ch1/intro.tex
tex/ch1/motivation.tex
....

In other words, our input is a list of paths that we want to do something to.

To find out what xargs does with these paths, a nice trick is to add echo before your command, like so:

$ git ls-files "*.tex" | xargs -n 1 echo sed -i "s/color/colour/g"
sed -i "s/color/colour/g" tex/ch1/intro.tex
sed -i "s/color/colour/g" tex/ch1/motivation.tex
....

The -n 1 argument will make xargs turn each line into a command of its own. The sed -i "s/color/colour/g" command will replace all occurrences of color with colour for the specified file.

Note that this only works if you don't have any spaces in your paths. If you do, you should use null terminated paths as input to xargs by passing the -0 flag. An example usage would be:

$ git ls-files -z "*.tex" | xargs -0 -n 1 sed -i "s/color/colour/g"

Which does the same as what we described above, but also works if one of the paths has a space in it.

This works with any command that produces filenames as output such as find or locate. If you do happen to use it in a git repository with a lot of files though, it might be more efficient to use it with git grep -l instead of git ls-files, like so:

$ git grep -l "color" "*.tex" | xargs -n 1 sed -i "s/color/colour/g"

The git grep -l "color" "*.tex" command will give a list of "*.tex" files containing the phrase "color".