sort unique urls from log

uniq | sort does not work: uniq removes contiguous duplicates.

The correct way is sort | uniq or better sort -u. Because only one process is spawned.

uniq needs its input sorted, but you sorted after uniq. Try:

$ sort -u < url.txt > urls.txt