Summary: Boe Prox shows us some tips about using runspace pools for multithreading.
Honorary Scripting Guy and Cloud and Datacenter Management MVP, Boe Prox, here today filling in for my good friend, The Scripting Guy. Today I continue a 3-part series about working with PowerShell runspaces. I’ll finish up with a fourth post that talks about a module I put together that makes working with runspaces as simple as using some similar commands that you are already familiar with. You might enjoy reading these posts first:
After spending the past couple of days working with runspaces and looking at how we can use parameters and arguments to supply variables to our runspaces, we are taking the next step in our journey by looking at runspace pools to do some multithreading with multiple commands. The process for working with runspace pools is similar to working with runspaces. There are a few minor differences in how we use the runspace pools with the PowerShell instance.
By using a runspace pool, we cannot only run multiple commands at the same time (which we could do using multiple runspaces), but we now have the capability to throttle the number of runspaces that are running concurrently. For example, I may have 50 items that I need to scan, but because of resource limits, I can only run six at a time. I queue up all of the objects, but only six will run at a given time. A new one will start when another one ends.
We can create the runspace pools by using the CreateRunspacepool() method from the [runspacefactory] type accelerator. We’ll provide some additional data during this construction that tells the runspace pools the limits for minimum runspaces and the maximum runspaces that can be allowed to run at a time (our throttle).
I am going to set up some parameters because I want to show how you can add that data (just like we did with runspaces):
$Parameters = @{
Param1 = 'Param1'
Param2 = 'Param2'
}
Next I will build out the runspace pools with a throttle of 10 and open the RunspacePool property so I can start working with it to add my script block.
#region Runspace Pool
[runspacefactory]::CreateRunspacePool()
$SessionState = [System.Management.Automation.Runspaces.InitialSessionState]::CreateDefault()
$RunspacePool = [runspacefactory]::CreateRunspacePool(
1, #Min Runspaces
5 #Max Runspaces
)
$PowerShell = [powershell]::Create()
#Uses the RunspacePool vs. Runspace Property
#Cannot have both Runspace and RunspacePool property used; last one applied wins
$PowerShell.RunspacePool = $RunspacePool
$RunspacePool.Open()
#endregion
With that accomplished, I can move forward to start running commands against my systems (or items) by creating a script block with my commands:
$jobs = New-Object System.Collections.ArrayList
1..50 | ForEach {
$PowerShell = [powershell]::Create()
$PowerShell.RunspacePool = $RunspacePool
[void]$PowerShell.AddScript({
Param (
$Param1,
$Param2
)
$ThreadID = [appdomain]::GetCurrentThreadId()
Write-Verbose "ThreadID: Beginning $ThreadID" -Verbose
$sleep = Get-Random (1..5)
[pscustomobject]@{
Param1 = $param1
Param2 = $param2
Thread = $ThreadID
ProcessID = $PID
SleepTime = $Sleep
}
Start-Sleep -Seconds $sleep
Write-Verbose "ThreadID: Ending $ThreadID" -Verbose
})
[void]$PowerShell.AddParameters($Parameters)
$Handle = $PowerShell.BeginInvoke()
$temp = '' | Select PowerShell,Handle
$temp.PowerShell = $PowerShell
$temp.handle = $Handle
[void]$jobs.Add($Temp)
Write-Debug ("Available Runspaces in RunspacePool: {0}" -f $RunspacePool.GetAvailableRunspaces())
Write-Debug ("Remaining Jobs: {0}" -f @($jobs | Where {
$_.handle.iscompleted -ne 'Completed'
}).Count)
}
There is a bit happening here that I need to explain. First, you probably noticed the ArrayList that I created to handle the jobs. I do this because as I add a new command to run in the PowerShell instance that uses RunspacePool, I need to call BeginInvoke() to kick off the command (if it is being helped due to throttling). That kicks back an Async object that I need to monitor for completion.
Also, I chose the script block that displays a simple object with some threading information. These work in a throttle of 10 items (threads) and will never use additional threads because the default ThreadOption for RunspacePool is ReuseThread.
Now that I have run all of my runspace jobs, I can sit back and wait for them to finish. I can check on them and when they are finished, I call EndInvoke() on each one to return the results of my scan.
#Verify completed
Write-Debug ("Available Runspaces in RunspacePool: {0}" -f $RunspacePool.GetAvailableRunspaces())
Write-Debug ("Remaining Jobs: {0}" -f @($jobs | Where {
$_.handle.iscompleted -ne 'Completed'
}).Count)
$return = $jobs | ForEach {
$_.powershell.EndInvoke($_.handle)
$_.PowerShell.Dispose()
}
$jobs.clear()
And for fun, I can look at the data to see that I only used a throttle of 10:
$return | Group ProcessID | Select Count, Name
$return | Group Thread | Select Count, Name
($return | Group Thread).Count
As we can see, my throttle of 10 was respected during the run against 50 items. I never went over that limit, and I was able to successfully perform my task.
Tomorrow I will wrap up my series about PowerShell runspaces by showing a module that I put together. It can allow you to use runspaces and runspace pools with ease by giving a familiar look and feel to them. Stay tuned!
I invite you to follow the Scripting Guy on Twitter and Facebook. If you have any questions, send email to the Scripting Guy at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. Until then, see ya!
Boe Prox, Windows PowerShell MVP and Honorary Scripting Guy