How would I use GNU Parallel for this while loop?

You don't use a while loop.

parallel "sortstuff.sh -a {} > sortedstuff-{}" <live_hosts

Note that this won't work if you have paths in your live_hosts (e.g. /some/dir/file) as it would expand to sortstuff.sh -a /some/dir/file > sortedstuff-/some/dir/file (resulting in no such file or directory); for those cases use {//} and {/} (see gnu-parallel manual for details):

parallel "sortstuff.sh -a {} > {//}/sortedstuff-{/}" <live_hosts

As an old-school "do one thing and do it well" Unix guy, I'd put the string substitution stuff into a wrapper script:

#!/bin/sh
sortstuff.sh -a "$1" > sortedstuff-"$1"

If you call it wrapper.sh, the parallel command to call it would be:

parallel wrapper.sh < live_hosts

Note that you don't need cat for this kind of thing, which saves an external program invocation.


You don't need parallel, since the body of the loop doesn't depend on previous iterations. Just start a new background process for each host.

while read host; do
    sortstuff.sh -a "$host" > sortedstuff-"$host" &
done < live_hosts
wait    # Optional, to block until the background tasks are done

parallel does make it easier to manage certain aspects, though; you can limit the number of jobs running in parallel more easily.