Is there a Windows equivalent to the Unix uniq?
The Sort-Object
cmdlet in PowerShell supports a -Unique
switch that does the same thing as uniq
:
Get-Content file.txt | Sort-Object -unique
Of course, owing to presence of aliases in PowerShell, you can also write:
type file.txt | sort -unique
Additionally, there is an undocumented /unique
switch in sort.exe
of Windows 10, so, this should work in Command Prompt:
type file.txt | sort /unique
There's ports of uniq that work identically to the gnu/coreutils versions. I personally use the variation from GOW but git for windows has a significantly newer version. No cygwin required though for the latter you need to look in /usr/bin
Since these packages also contain cat, sort and uniq - your workflow should be mostly identical, and cat file.txt |sort | uniq
should work mostly identically
You can easily write the command "uniq" by yourself. Save this in a batch file "uniq.cmd" somewhere in your %path% can find it (e.g. in %windir%\system32). This version is NOT case sensitive:
@echo off
setlocal DisableDelayedExpansion
set "prev="
for /f "delims=" %%F in ('sort %*') do (
rem "set" needs to be done without delayed expansion
set "line=%%F"
setlocal EnableDelayedExpansion
set "line=!line:<=<!"
if /i "!prev!" neq "!line!" echo(!line!
set "prev=!line!"
endlocal
)
This works with "uniq mytextfile" as well as "cat mytextfile | uniq"; as all input and arguments are simply passed to the sort command.
Starting with Windows 7, you may want a really case sensitive version (the difference ist undocumented switch "sort /C" and no "if /i"):
@echo off
setlocal DisableDelayedExpansion
set "prev="
for /f "delims=" %%F in ('sort /C %*') do (
rem "set" needs to be done without delayed expansion
set "line=%%F"
setlocal EnableDelayedExpansion
set "line=!line:<=<!"
if "!prev!" neq "!line!" echo(!line!
set "prev=!line!"
endlocal
)