What's a good way to filter a text file to remove empty lines?
You can use grep's -v
(invert match) mode to do this:
grep -v '^$' old-file.csv > new-file.csv
Note that those need to be different files, because of how shell redirects work. The output file is opened (and emptied) before the input file is read. If you have moreutils (not by default on Mac OS X), you can use sponge
to work around this:
grep -v '^$' file.csv | sponge file.csv
But of course, then you have a harder time going back if something goes wrong.
If you "blank lines" actually may contain spaces (it sounds like they do), then you can use this instead:
egrep -v '^[[:space:]]*$' old-file.csv > new-file.csv
That will ignore blank lines as well as lines containing only whitespace. You can of course do the same sponge
transformation on it.
The easiest option is just grep .
. Here, the dot means "match anything", so if the line is empty, it is not matched. Otherwhise it prints the whole line as is.
To remove empty lines, in place, with ksh93:
sed '/./!d' file 1<>; file
The <>;
redirection operator is specific to ksh93 and is the same as the standard <>
operator except that ksh truncates the file after the command has terminated.
sed '/./!d'
is a convoluted way to write grep .
, but unfortunately GNU grep at least complains if its stdout points to the same file as its stdin. You'd say one could write:
grep . file | cat 1<>; file
But unfortunately, there's a bug in ksh93 (at least my version (93u+)), in that the file seems to be truncated to zero length in that case.
grep . file | { cat; } 1<>; file
Seems to work around that bug, but now, it's far more convoluted than the sed command.