Azure Automation powershell runbook hung up issue

Question

I am creating an Azure Automation account having a powershell runbook. The script is simply looping over each VM in a resource group and deleting all files under a particular folder.

$ResourceGroup = '<<myResourceGroup>>'
$TargetDir = '<targetDirectoryPath/.'
$TargetCommand = 'rm -rfv ' + $TargetDir

try
{
    $message = Connect-AzAccount -Identity  
    
    $myAzureVMs = Get-AzVM -ResourceGroupName $ResourceGroup -status | Where-Object {$_.PowerState -eq "VM running" -and $_.StorageProfile.OSDisk.OSType -eq "Linux"}
    
    if($myAzureVMs.Name.Count -gt 1)
    {
        For($val = 0; $val -le $myAzureVMs.Name.Count-1; $val++)
        { 
           # the script hangs here
            $message = Invoke-AzVMRunCommand -ResourceGroupName $ResourceGroup -Name $myAzureVMs.Name[$val] -CommandId 'RunShellScript' -ScriptString $TargetCommand 
        }    
    }

    Write-Output "DONE !!"
}
catch
{
    Write-Error -Message $_.Exception
    throw $_.Exception
}

The script is working fine but the issue is if a VM is not ok or has some connectivity problem then the script hangs just there. No error or exception is thrown. It just keeps waiting and eventually times out after hours.

My question is there any way to check if the VM can be connected or not (or any better solution) and then only the script should execute. I have also tried putting the Invoke-AzVMRunCommand inside a try-catch block to eat up the exception and to move to the next VM but in vain. It's simply hangs on to that VM and no exception thrown.

score 0 · Answer 1 · answered Jan 20 '23 at 07:00

Before collecting the "VM status" or relevant virtual machine details, I would suggest using the
start-AzVM Azure PowerShell command to check whether VM is connected.

The script should then be called based on the connectivity status it returns. You can avoid waiting time by doing so while running the remaining script.

$vmstatus = start-AzVM -Name <VMName> -ResourcegroupName <resourcegroupname>
if($vmstatus.status -eq "succeeded") {
write-host "connected"
}

Output:

enter image description here

Alternatively, check the Test-AzNetworkWatcherConnectivity by enabling Network Watcher in your environment.

score 0 · Accepted Answer · answered Jan 23 '23 at 05:00

Solved. Actually the run command utilizes the VM's agent to execute the task and in some VMs this agent is not properly ready and that's why the hung up. So, added an additional checking to check the VM agent status on each VM before executing the operation and that solved the problem.

Azure Automation powershell runbook hung up issue

2 Answers2