I have a script using workflows and Foreach -Parallel -ThrottleLimit 15 which works fine if the code inside the loop is something simple like write-output or some basic math and a sleep statement, but when I swap in the actual logic (updating product for a particular client) it caps at 5 running simultaneously.
Unfortunately, I can't post the full source but at a high level the update process reads from some config files occasionally, writes to log files (separate logs for each thread), reads/writes to a couple of databases, nothing to cause the CPU/RAM/IO to max or anything though and to the best of our knowledge there is no contention going on between threads over these resources.
Things We've looked into:
- Powershell 3 had a hard limit of 5 (We are using Powershell 5 and are able to run more than 5 at a time when replacing the actual work with simple math and a write-output)
- CPU/RAM/Disk IO all appear to be pretty stable (not maxed out or spiking a lot)
- Look for MS documentation on this param (not much available and doesn't mention any strange behavior that I've seen)
I'm at a loss as to where else to look to troubleshoot this. At first we thought maybe the ThrottleLimit was a maximum and the actual number was dynamic based on resources, but our limited testing seemed to indicate that it uses the full 15 even when CPU and Memory are pretty high. Also, the very first iteration is capped at 5, long before any resources are actually being used heavily (there is a start-sleep of 15 minutes right away in the update script as we allow a warning to our users to finish what they are doing before continuing to take down the DB for the update.)
Does anyone have any insight into what else we can check or look into that might cause this? Our previous parallel logic was using Jobs and I wanted to avoid returning to that logic when this seems like it should work perfectly fine.
