1

I have a script that gets a list of VMs from vROps, and iterates through them in a ForEach loop. My Azure Automation sandbox runs out of memory after about 15 minutes, but if I put a Clear-Variable at the end of the ForEach loop, it runs until it runs out of time at 3 hours (sigh)...

After googling to no avail, I cannot figure out why it takes so much less memory to clear a variable, and then recreate it during the next iteration, compared to just overwriting it, which seems to slowly take up more and more memory. So, my question is, why is this?

Lars Panzerbjrn
  • 196
  • 3
  • 12
  • 4
    Show us your code – Mathias R. Jessen Apr 23 '21 at 17:18
  • I would like to also add to his question, when is good to call `[gc]::Collect()` or if ever is in PowerShell. – Santiago Squarzon Apr 23 '21 at 17:21
  • 1
    This is not specifically a PS issue, but Windows Memory management. Though you are reusing the variable, it's not in the same place in memory use reuse. Garbage Collection (gc) is not immediate (clearing used memory maps), though performant. So, you end up with multiple blocks occupied. Just like on a hard drive, delete does not mean delete, it means to mark this as reusable, and at Layer 1 of OS, stuff gets move/fragmented as needed, but you never will serilize back to the same sector. – postanote Apr 23 '21 at 19:49

1 Answers1

2

Continuing from my comment.
For the most part, your post could be a duplicate or very close to this SO Q&A

Powershell - "Clear-Item variable:" vs "Remove-Variable"

Some specific points of debate.

... reference is the last reference to the object then the GC will determine when the memory for said object is collected. However, if you no longer need the variable then use Remove-Variable to also allow the memory associated with the System.Management.Automation.PSVariable object to be eventually collected as well.

... Remember, Powershell is all .NET, with its famous memory management. Always control your scope and make sure variables get out of scope as soon as they are not needed. For example, if temporary variables are needed inside loops, they will invalidate at loop's end automatically

... To clear variable data/content is to remove all variables running in the current session using:

Remove-Variable -Name * -ErrorAction SilentlyContinue This removes all variables immediately. In fact, I add this to the end of some of my scripts, so that I can be sure that running another script with potentially the same name, will not have new data added and cause undesired results.

Thus, because of items like the above, historically, I capture default session stuff on startup...

$AutomaticVariables    = Get-Variable
$AutomaticVModules     = Get-Module
$AutomaticAliases      = Get-Alias
... and a few more

... and in my code execute my Clear-ResourceEnvironment function after each run, which does...

  • Clear resource interop
  • Clear only variables created/used during the session
  • Clear static credential store, if the switch is used
  • Clear only modules loaded during the session
  • Clear all PSSessions
  • Force start .Net garbage collection

... to alleviate such impacts. Now, of course, loading that much also has footprint impacts.

postanote
  • 15,138
  • 2
  • 14
  • 25
  • So if I understood correctly, `[gc]::Collect` is not useful and can be replace with `Remove-Variable` in PowerShell – Santiago Squarzon Apr 23 '21 at 20:33
  • 1
    `GC` will happen regardless of the `Remove-Variable`, because, well, you know, Windows. ***;-}*** I created my function to do both by design, just because I want to know I have a clean environment per each run, rather than waiting for some other process, that I don't control to do it. – postanote Apr 23 '21 at 20:39
  • I assume this is the post for reference for anyone reading. https://stackoverflow.com/questions/54965564/clear-memory-buffer-of-powershell-running-endless-loop-every-3-seconds :) – Santiago Squarzon Apr 23 '21 at 20:43
  • That is one I responded to in the past of course and was selected as the OP's answer for their use case, but the function shown is just an earlier version of the one I have in my library and use today. I can get you that if you want to see it. – postanote Apr 23 '21 at 20:51
  • I was searching for your GitHub on your profile but didn't find any link lol. Yeah that would be great :D – Santiago Squarzon Apr 23 '21 at 20:55
  • I did create one a while back, github.com/postanote, but have not posted anything to it yet; because I am still deciding how I want to set that up for ease of discovery/contribution/use, etc. – postanote Apr 23 '21 at 21:22
  • Thanks. That post was one of the ones I looked at before posting myself, and was the first I heard of GC, which I implemented before writing this post, and which allowed my script to at least run until it runs out of time... It did touch on some of the same things, but didn't really address my particular case. Between your other comment, and this answer, I guess I'll just have to accept taht it is a Powershell/.Net limitation, and get used to using Remove-Variable in scripts that go into Azure Automation. – Lars Panzerbjrn Apr 24 '21 at 10:39