Running tasks parallel in powershell
According to Get-Help about_Foreach-Parallel
, ForEach -Parallel...
will process the entire scriptblock in parallel for each item, but the commands in the scriptblock will be processed sequentially (though presumably they will be parallelized if bracketed with Parallel {...}
). However, your script must be a PowerShell workflow for this to be accepted; the Parallel
and Sequence
keywords are only effective within workflows.
Powershell 7 introduces foreach-object -parallel
:
https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature/
Your script would then say
$files | ForEach-Object -parallel {
[Do something]
[Do something]
[Do something]
}
You might look into Jobs or runspaces. Here is an example of Jobs:
$block = {
Param([string] $file)
"[Do something]"
}
#Remove all jobs
Get-Job | Remove-Job
$MaxThreads = 4
#Start the jobs. Max 4 jobs running simultaneously.
foreach($file in $files){
While ($(Get-Job -state running).count -ge $MaxThreads){
Start-Sleep -Milliseconds 3
}
Start-Job -Scriptblock $Block -ArgumentList $file
}
#Wait for all jobs to finish.
While ($(Get-Job -State Running).count -gt 0){
start-sleep 1
}
#Get information from each job.
foreach($job in Get-Job){
$info= Receive-Job -Id ($job.Id)
}
#Remove all jobs created.
Get-Job | Remove-Job
In the above code I have it where each $file
is running in parallel with eachother (Up to 4 running simultaneously).
EDIT: In response to the comments, here is some documentation about scriptblocks. The short reason about why you must include the parameter is because unlike PowerShell functions, scriptblocks can't specify parameters outside of the braces {}.