Summary: Microsoft Scripting Guy, Ed Wilson, talks about how to configure Windows PowerShell memory availability for specialized applications.
Hey, Scripting Guy! I really need your help. We are doing something that perhaps Windows PowerShell cannot do. At least, this is the way it seems. We have a huge file share, and we are parsing through it with a series of Select-String commands to find specific types of things. We are using Get-ChildItem to obtain files for us to parse, and we have filtered it out as much as is possible. The thing is, that when I say huge, I mean really, really huge. Absolutely GINORMOUS type of huge.
Anyway, we are perfectly resigned to the fact that Windows PowerShell will take some time to go through this parsing effort, and we have obviously tested everything on mock (much smaller) data prior to turning this thing loose. The problem is that Windows PowerShell runs for an hour or so, and then it stops with a System.OutOfMemoryException error message. It is bad enough that it happens, but it is horrible that it takes so long to occur. We make changes, reboot the server, wait for another hour, and boom!—it happens again. We have spent an entire week trying to make this work, and you are our last hope. I searched the Hey, Scripting Guy! blog, but I did not find anything helpful. So now’s your chance to be a real hero.
—AP
Hello AP,
Microsoft Scripting Guy, Ed Wilson, is here. Today is a great day. I got up, and fixed some nice English Breakfast tea with a bit of organic orange rind, some peppermint and spearmint leaves, a bit of crushed cinnamon stick, and a touch of lemon grass. I must say, it is a very refreshing cup of tea. Yesterday I had an awesome session with my mentee, Ashley McGlone. I am real proud of everything he has accomplished so far. So the week is going along perfectly. I am looking forward to this Thursday (August 1, 2013). We are having the Windows PowerShell User Group meeting in Charlotte, North Carolina. It will be awesome. With everything moving along smoothly, I thought I would take some time to try to catch up a bit with questions such as yours that are emailed to scripter@microsoft.com.
Configuring memory for Windows PowerShell
To configure memory resources for Windows PowerShell, you must launch Windows PowerShell with Admin rights. If you attempt to do anything on the WSMAN: drive as a normal user, you will receive the following “Access is denied” error message:
In addition to Admin rights, the WinRM service must be running. In Windows PowerShell 3.0 in Windows 8, this service starts on demand. Therefore, the first attempts to access the WinRM drive will result in a prompt to start the WinRM service. I use the Get-Service cmdlet to ensure that everything started properly:
PS C:\> get-service *win*
Status Name DisplayName
------ ---- -----------
Running WinDefend Windows Defender Service
Running WinHttpAutoProx... WinHTTP Web Proxy Auto-Discovery Se..
Running Winmgmt Windows Management Instrumentation
Running WinRM Windows Remote Management (WS-Manag..
Check and set the machine-wide setting
The first thing to do is to check and set the machine-wide memory setting. To do this, I navigate to WsMan:\Localhost\Shell in my Windows PowerShell console. I then use the Get-ChildItem cmdlet (dir is alias) to see my settings for everything. This is shown here:
PS C:\> sl WSMan:\localhost\Shell
PS WSMan:\localhost\Shell> dir
WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Shell
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String AllowRemoteShellAccess true
System.String IdleTimeout 7200000
System.String MaxConcurrentUsers 10
System.String MaxShellRunTime 2147483647
System.String MaxProcessesPerShell 25
System.String MaxMemoryPerShellMB 1024
System.String MaxShellsPerUser 30
Set MaxMemoryPerShellMB
To make the change, I use the Set-Item cmdlet and change the value of MaxMemoryPerShellMB from 1 GB to 2 GB. This technique is shown here:
Set-Item .\MaxMemoryPerShellMB 2048
Now I use the Up arrow and change Get-Item to Set-Item. This command and its output are shown here:
Note I am already in WsMan:\LocalHost\Shell when I run the Set-Item command. If you do not want to navigate to the folder first, you can use this command:
Set-Item WSMan:\localhost\Shell\MaxMemoryPerShellMB 2048
I notice that a warning appears that states I also need to change memory settings for plug-ins. (This is true in Windows PowerShell 3.0.) Therefore, I navigate to the plug-ins directory to make those changes. But before I make any changes, I notice there are several plug-ins listed:
PS WSMan:\localhost\Plugin> dir
WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Plugin
Type Keys Name
---- ---- ----
Container {Name=Event Forwarding Plugin} Event Forwarding Plugin
Container {Name=microsoft.powershell} microsoft.powershell
Container {Name=microsoft.powershell.workf... microsoft.powershell.workflow
Container {Name=microsoft.powershell32} microsoft.powershell32
Container {Name=microsoft.windows.serverma... microsoft.windows.servermanag...
Container {Name=WMI Provider} WMI Provider
The thing that is confusing is that I need to make a memory change for each plug-in endpoint configuration that I target from the client. Luckily, I happen to know that the default Windows PowerShell endpoint is Microsoft.PowerShell, and that is the only one I need to change. I type the following command:
Set-Item .\microsoft.powershell\Quotas\MaxConcurrentCommandsPerShell 2048
The command results in a warning that states I need to restart WinRM and that the value for the plug-in will only work if it is less than or equal to the value for the global memory setting. Here is the command and the output:
Note I was in the Wsman:\LocalHost\Plugin directory when I ran the command to set the memory for the plug-in. If you do not want to navigate to the location, use this command instead:
Set-Item WSMan:\localhost\Plugin\Microsoft.PowerShell\Quotas\MaxMemoryPerShellMB 2048
I use the Get-Item cmdlet to ensure that the new value took. Here is the command I use:
PS WSMan:\localhost\Plugin> get-Item .\microsoft.powershell\Quotas\MaxMemoryPerShellMB
WSManConfig:
Microsoft.WSMan.Management\WSMan::localhost\Plugin\microsoft.powershell\Quotas
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String MaxMemoryPerShellMB 2048
Restart the WinRM service
Now I need to restart the WinRM service. To do this, I use the Restart-Service cmdlet. The command is shown here:
Restart-Service winrm
Just for fun, I close the Windows PowerShell console, and then reopen it. I rerun my Get-Item commands to see if anything has reverted. As indicated in the image that follows, everything is groovy.
AP, that is all there is to using the WSMAN drive to configure Windows PowerShell memory. Join me tomorrow when I will talk about more cool stuff.
I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.
Ed Wilson, Microsoft Scripting Guy