Why does the command shuf file > file leave an empty file, but similar commands do not?
The problem is that > example.txt
starts writing to that file, before shuf example.txt
starts reading it. So as there was no output yet, example.txt
is empty, shuf
reads an empty file, and as shuf
makes no output in this case, the final result stays empty.
Your other command may suffer from the same issue. > example.txt
may kill the file before cat example.txt
starts reading it; it depends on the order the shell executes those things, and how long it takes cat
to actually open the file.
To avoid such issues entirely, you could use shuf example.txt > example.txt.shuf && mv example.txt.shuf example.txt
.
Or you could go with shuf example.txt --output=example.txt
instead.
The package moreutils has a command sponge
:
sponge reads standard input and writes it out to the specified file.
Unlike a shell redirect, sponge soaks up all its input before opening
the output file. This allows constricting pipelines that read from and
write to the same file.
with that way you can do:
shuf example.txt | sponge example.txt
(unfortunately the moreutils package also has a util named parallel
that is far less useful than gnu parallel. I removed the parallel
installed by moreutils)
You are just quite lucky running
cat example.txt | shuf > example.txt
doesn't empty example.txt
like this command is doing.
shuf example.txt > example.txt
Redirections are performed by the shell before the commands are executed and pipeline components are executed concurrently.
Using the -o
/ --output
option would be the best solution with shuf
but if you like taking (very slight) risks, here is a non traditional way to avoid the processed file to be truncated before being read:
shuf example.txt | (sleep 1;rm example.txt;cat > example.txt)
and this simpler and faster one, thanks to Ole's suggestion:
(rm example.txt; shuf > example.txt) < example.txt