Keep x number of files and delete all others - Powershell

You can sort by CreationTime descending and skip the first 10. If there are less than 10 files it will not remove any.

gci C:\temp\ -Recurse| where{-not $_.PsIsContainer}| sort CreationTime -desc| 
    select -Skip 10| Remove-Item -Force

Updated

$path = "C:\TestDir"

# Create 12 test files, 1 second after each other
1..12 | % {
    Remove-Item -Path "$path\$_.txt" -ea SilentlyContinue
    $_ | Out-File "$path\$_.txt"
    Start-Sleep -Seconds 1
}

$files = Get-ChildItem -Path $path -Recurse | Where-Object {-not $_.PsIsContainer}
$keep = 10
if ($files.Count -gt $keep) {
    $files | Sort-Object CreationTime | Select-Object -First ($files.Count - $keep) | Remove-Item -Force -WhatIf
}

Remove the -WhatIf parameter to Remove-Item when your ready to delete for real.

@Andy: My rep is too low to comment so posting an answer. If you are getting strange results, it may be because you are looking in Windows Explorer which by default shows the Last Modified date instead of the Creation Date. To get results consistent with that shown in Windows Explorer, swap LastWriteTime for CreationTime like so: gci C:\temp\ -Recurse| where{-not $_.PsIsContainer}| sort LastWriteTime -desc | select -Skip 10| Remove-Item -Force