How can I bulk rename files using PowerShell?

I would first filter by 5* so you only process names that start with 5. Also, in this case since tf.exe isn't a PowerShell cmdlet, you don't want to use a scriptblock to determine a new name. Just use a grouping expression like so:

dir -filter 5* | foreach { tf rename $_ ($_.Name -replace '^5', '2.3.2')}

BTW, when you are trying to debug parameter passing to a native EXE like this it is immensely helpful to use the echoargs.exe utilty from the PowerShell Community Extensions. This is what it told me about your original approach:

6# dir -filter 5* | foreach { echoargs rename $_ { $_.Name -replace '^5', '2.3.2' } }
Arg 0 is <rename>
Arg 1 is <5foo.txt>
Arg 2 is <-encodedCommand>
Arg 3 is <IAAkAF8ALgBOAGEAbQBlACAALQByAGUAcABsAGEAYwBlACAAJwBeADUAJwAsACAAJwAyAC4AMwAuADIAJwAgAA==>
Arg 4 is <-inputFormat>
Arg 5 is <xml>
Arg 6 is <-outputFormat>
Arg 7 is <text>

Notes:

• TFS has native cmdlets -- no need for tf.exe in most cases.
• The time complexity of workspace operations depends on the number of pending renames already in the workspace. In TFS 2005/2008 it's significantly worse than linear. Bottom line, you should really consider batching up renames into multiple checkins if you have a large # of items, otherwise every single "tf rename" (or New-TfsPendingChange -Rename if using the cmdlets) will start taking minutes.