Using Windows/DOS shell/batch commands, how do I take a file and only keep unique lines?

@echo off
setlocal disabledelayedexpansion
set "prev="
for /f "delims=" %%F in ('sort uniqinput.txt') do (
  set "curr=%%F"
  setlocal enabledelayedexpansion
  if "!prev!" neq "!curr!" echo !curr!
  endlocal
  set "prev=%%F"
)

What it does: sorts the input first, and then goes though it sequentially and outputs only if current line is different to previous one. It could have been even simpler if not for need to handle special characters (that's why those setlocal/endlocal are for).
It just echoes lines to stdout, if you want to write to file do (assuming you named your batch myUniq.bat) myUniq >>output.txt

There's no easy way to do that from the command line without an additional program.

uniq will do what you want.

Or you can download CoreUtils for Windows to get GNU tools. Then you can just use sort -u to get what you want.

Either one of those should be callable from a batch file.

Personally though, if you need to do a lot text manipulation like that I think you'd be better off getting Cygwin. Then you'd have easy access to sort, sed, awk, vim, etc.

Run PowerShell from the command prompt.

Assuming the items are in a file call fruits.txt, the following will put the unique lines in uniques.txt:

type fruits.txt | Sort-Object -unique | Out-File uniques.txt