How to Increase number of cuncurrent powershell instance on VM?

Question

I am trying to trigger a powershell workflow which should spin 10 threads in parallel. I am using PS version 4. Code -

Workflow CreateVMs
{  
    $i = 0
    #somecodehere...
    foreach -parallel -throttlelimit 10($i in 0..30)
    {
      #somemorecodehere...
      # INVOKING BATCH FILE USING POWERSHELL
     }
}

I am invoking batch files using powershell inside my inline script. What I observed in the task manager is, only 5 threads were active at a time-

May be the next thread getting picked up only one of the five is completed. I never saw more than 5 ps instance. However, When i checked the ps sessions allowed per user it is far more than 5.

How can I spin 10 threads of PS in parallel in a PS workflow. What am I missing here ?

As long as the number of parallel activities doesn't exceed the throttle limit, Workflow Foundation will manage the thread/activity pool at it's own whim, there's no way for you to force it to "go faster" — Mathias R. Jessen, Apr 10 '17 at 14:13
Not sure if this would work for you, but have you considered using the Start-Job cmdlet to accomplish what you are trying to do? https://msdn.microsoft.com/en-us/powershell/reference/5.1/microsoft.powershell.core/start-job — Caleb Seelhoff, Jun 30 '17 at 17:16
See this thread http://www.madwithpowershell.com/2014/03/stuff-you-didnt-know-about-foreach-in.html and a comment which says `It turns out the behavior only occurs if there is a InlineScript{} block in the ForEach. But, if there is, it does limit the execution to 5 threads.` See if you can make some sense out of that comment — Tarun Lalwani, Nov 27 '17 at 09:06
Those are processes, not threads. You can see new threads spawning in something like process monitor. — js2010, Aug 02 '19 at 14:51

score 1 · Answer 1 · answered Nov 28 '17 at 19:32

Given the following sample script:

Workflow Test-Workflow
{
    foreach -Parallel -ThrottleLimit 10 ( $Number in 1..20 )
    {
        $RandomSeconds = Get-Random -Minimum 10 -Maximum 60

        InlineScript
        {
            $String = '{0:s}: Starting number {1:D2} for {2:D2} seconds' -f (Get-Date),$Using:Number,$Using:RandomSeconds
            Write-Host -Object $String
        }

        Start-Sleep -Seconds $RandomSeconds

        InlineScript
        {
            $String = '{0:s}: Stopping number {1:D2} after {2:D2} seconds' -f (Get-Date),$Using:Number,$Using:RandomSeconds
            Write-Host -Object $String
        }
    }
}
Test-Workflow

You will get output similar to the following:

2017-11-28T13:27:34: Starting number 09 for 25 seconds
2017-11-28T13:27:34: Starting number 10 for 36 seconds
2017-11-28T13:27:34: Starting number 08 for 53 seconds
2017-11-28T13:27:35: Starting number 06 for 17 seconds
2017-11-28T13:27:35: Starting number 07 for 28 seconds
2017-11-28T13:27:35: Starting number 05 for 33 seconds
2017-11-28T13:27:35: Starting number 04 for 49 seconds
2017-11-28T13:27:35: Starting number 02 for 18 seconds
2017-11-28T13:27:35: Starting number 03 for 47 seconds
2017-11-28T13:27:35: Starting number 01 for 45 seconds
2017-11-28T13:27:52: Stopping number 06 after 17 seconds
2017-11-28T13:27:55: Starting number 11 for 49 seconds
2017-11-28T13:27:55: Stopping number 02 after 18 seconds
2017-11-28T13:27:55: Starting number 12 for 55 seconds
2017-11-28T13:28:00: Stopping number 09 after 25 seconds
2017-11-28T13:28:00: Starting number 13 for 37 seconds
2017-11-28T13:28:03: Stopping number 07 after 28 seconds
2017-11-28T13:28:03: Starting number 14 for 46 seconds
2017-11-28T13:28:08: Stopping number 05 after 33 seconds
2017-11-28T13:28:08: Starting number 15 for 48 seconds
2017-11-28T13:28:11: Stopping number 10 after 36 seconds
2017-11-28T13:28:11: Starting number 16 for 57 seconds
2017-11-28T13:28:21: Stopping number 01 after 45 seconds
2017-11-28T13:28:21: Starting number 17 for 22 seconds
2017-11-28T13:28:22: Stopping number 03 after 47 seconds
2017-11-28T13:28:22: Starting number 18 for 39 seconds
2017-11-28T13:28:24: Stopping number 04 after 49 seconds
2017-11-28T13:28:24: Starting number 19 for 34 seconds
2017-11-28T13:28:28: Stopping number 08 after 53 seconds
2017-11-28T13:28:28: Starting number 20 for 58 seconds
2017-11-28T13:28:37: Stopping number 13 after 37 seconds
2017-11-28T13:28:43: Stopping number 17 after 22 seconds
2017-11-28T13:28:44: Stopping number 11 after 49 seconds
2017-11-28T13:28:49: Stopping number 14 after 46 seconds
2017-11-28T13:28:50: Stopping number 12 after 55 seconds
2017-11-28T13:28:56: Stopping number 15 after 48 seconds
2017-11-28T13:28:58: Stopping number 19 after 34 seconds
2017-11-28T13:29:01: Stopping number 18 after 39 seconds
2017-11-28T13:29:08: Stopping number 16 after 57 seconds
2017-11-28T13:29:26: Stopping number 20 after 58 seconds

As you can see from the Output, 10 instances of the loop are running concurrently even though I do not have 10 powershell.exe instances running. You can use InlineScript like in my example above to visually see when your workflow is starting and/or stopping an instance of your loop.

@Atf I was using the Inline script blocks to display the start and stop time of each iteration. As you can see from the output, it shows that there are 10 concurrent iterations running. — Shawn Esterman, Nov 29 '17 at 19:10
Thanks for your answer. But in your case the commands inside the inline script are not creating any process. It is a simple write-host thing. I kept invoking different batch files using powershell inside my inline script and the behavior is the same. My problem is still only 5 threads of powershell can be seen in task manager which is actual. EDITED QUESTION TO CLARIFY — Aatif Akhter, Nov 29 '17 at 19:19
@Atf WELL you could use use the `InlineScript` feature to write out put to a console and display that 10 iterations are actually running. I didn't write the PowerShell foreach function, but I would suspect it actually uses a runspace instead of creating a bunch of powershell.exe instances. — Shawn Esterman, Nov 29 '17 at 19:21
It is still the same. May be the same thing @tarun Lalwani has commented. Probably I should try the same with PS version 5.1 — Aatif Akhter, Nov 29 '17 at 19:25

js2010 · Answer 2 · 2019-08-03T02:31:18.760

If I understand your question correctly, yes, there seems to be a difference between launching powershell processes and running builtin powershell (emulated) commands in a workflow. It seems to limit the process count to 5 in task manager. Maybe you can update your example with what you are running. Maybe you'd prefer jobs or start-process.

 workflow work { foreach -parallel ($i in 1..10) { powershell "sleep 10; echo done $i" }  }
 measure-command { work } | fl seconds

 Seconds           : 35

 workflow work { foreach -parallel ($i in 1..10) { sleep 10; echo done $i }  }
 measure-command { work } | fl seconds

 Seconds           : 14

EDIT:

Uhh, it can be done. It's somewhat involved. You have to be administrator, and psremoting has to be enabled. Although I saw more processes run, it actually took longer to finish.

# all default to 5
$wfopt = New-PSWorkflowExecutionOption -MaxSessionsPerWorkflow 20 -MaxSessionsPerRemoteNode 20 -MaxActivityProcesses 20

Register-PSSessionConfiguration -Name myworkflow -SessionTypeOption $wfopt -SessionType Workflow -Force

work -PSConfigurationName myworkflow

You can use Set-PSSessionConfiguration to modify it, Get-PSSessionConfiguration to check it, and UnRegister-PSSessionConfiguration to remove it.

You could modify the default config too, Microsoft.PowerShell.Workflow, but the changes are permanent.

score 0 · Answer 3 · answered Aug 02 '19 at 14:02

I using PS v5.1 and still have the same limitation. Is there a way around these limits on some newer version of PS? Maybe on PS6?

Name                           Value
----                           -----
PSVersion                      5.1.17134.858
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.17134.858
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Follow the code and results:

foreach -parallel -throttlelimit 10 ($i in 0..20)
{
    "start #$i"
    InlineScript { "    inline #$using:i" }
}

start #9
start #8
start #7
start #6
start #5
start #4
start #3
start #2
start #1
start #0
    inline #
    inline #
    inline #
    inline #
    inline #
start #10
start #11
start #12
start #13
start #14
    inline #
    inline #
    inline #
    inline #
    inline #
start #15
start #16
start #17
start #18
start #19
    inline #
    inline #
    inline #
start #20
    inline #
    inline #
    inline #
    inline #
    inline #
    inline #
    inline #
    inline #

Please create a new question instead of adding it to this thread. — Dave, Aug 02 '19 at 14:22

How to Increase number of cuncurrent powershell instance on VM?

3 Answers3