Unique sorting: Redirect output to the same file

You can use sponge from moreutils package:

LC_ALL=C sort -u filename | sponge filename

You also don't need pipe to uniq, since when sort has -u option to unique lines when sorting.

Note that on GNU system with UTF-8 locales, sort -u or sort | uniq didn't give you unique lines, but the first from sequence of lines which sort the same in current locale.

$ printf '%b\n' '\U2460' '\U2461' | LC_ALL=en_US.utf8 sort | LC_ALL=en_US.utf8 uniq
①

gave you only ①. Changing locale to C force the sorting order based on the byte values:

$ export LC_ALL=C
$ printf '%b\n' '\U2460' '\U2461' | LC_ALL=C sort | LC_ALL=C uniq
①
②

You don't need any extra command like cat and uniq and also without using rm command and mv command to removing and renaming the filename. just use simple command.

sort -u filename -o filename

 -u, --unique
        with -c, check for strict ordering; without -c, output only  the
        first of an equal run

 -o, --output=FILE
        write result to FILE instead of standard output

How it work?

sort command sorts your filename and with -u option, removes duplicate lines from it. then with -o option writes the output to the same file with in place method.

Your suggested example (below) doesn't work because you'd actually be reading from and writing to the same file simultaneously.

$ cat filename | sort | uniq > filename

The idea with a pipe or redirect is that the command on the left and right hand side of each pipe or redirect run simultaneously, in parallel. The command on the right processes information as it is handed over to it from the command on the left, while the command on the left is still running.

In order for your scenario to work, the command that reads from the file would need to finish before the command that writes to the file begins. In order for this to work you would need to redirect the output into a temporary location first, then once that's finished, send it from the temporary location back into the file.

A better way to do this is basically like in your former example, where you redirect to a temporary file then rename that file back to the original (except that you don't need to delete the file first, because moving deletes any existing target).

$ cat filename | sort | uniq > result
$ mv -f result filename

You could also save it into a string variable, except that only works when data is small enough to all fit in memory at once.