3

I am trying to generate a 1000 Azure VMs(yes I know of the cost) for a large job. This is being done in a PowerShell script (see script below) asynchronously so charges will not be incurred while waiting for all the VMs to spin up.

In the script if Wait-Job and Receive-Job include when the script runs all the requested VMs are created, but if Wait-Job and Receive-Job are commented out all the VMs are not created. It seems random what VMs are getting created.

Can anyone see what I'm doing wrong with this script?

$VMCount = 5
$Location = 'East US'
$Image = 'MyImage'
$AdminPassword = 'XXXXXXXXXX'
$LinuxUser = 'MyUser'
$InstanceSize = 'ExtraSmall' #extra small only for testing

$CloudServiceName = 'NewAzureMachinePrefix' #this is changed each time to something unique

for ($i = 1; $i -le $VMCount; $i++)
{
$jobId = Start-Job -ArgumentList $CloudServiceName$i, $Location, $Image, $AdminPassword, $LinuxUser, $InstanceSize -ScriptBlock {
    param($ServiceName, $Loc, $Img, $Password, $User, $Size) 
    New-AzureVMConfig -Name $ServiceName -InstanceSize $Size -ImageName $Img | 
    Add-AzureProvisioningConfig -Linux -LinuxUser $User -Password $Password | 
    Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel $ServiceName -LUN 0 | 
    Remove-AzureEndpoint 'SSH' | 
    Add-AzureEndpoint -Protocol tcp -PublicPort 22 -LocalPort 22 -Name 'SSH' | 
    Set-AzureSubnet 'MySubnet' | 
    New-AzureVM -ServiceName $ServiceName -AffinityGroup 'MyGroup' -VNetName 'MyNet'
}

Write-Output $CloudServiceName$i
Wait-Job $jobId
Receive-Job $jobId
}
Rick Gittins
  • 1,138
  • 1
  • 8
  • 24

2 Answers2

2

I figured out what is going on after a few emails from our Microsoft Rep. When creating a new virtual machine in Azure using an Affinity Group and/or Virtual Network an exclusive lock is created. This exclusive lock does not allow more than one request to access the Affinity Group and/or Virtual Network.

Rick Gittins
  • 1,138
  • 1
  • 8
  • 24
  • Were you able to come up with a solution that didn't require Wait-Job? – Glenn Nov 04 '14 at 15:44
  • 1
    The code in the script block (Start-Job) creates the virtual machine for you. You don't need to run Wait-Job and Receive-Job to get the virtual machine created. – Rick Gittins Nov 04 '14 at 20:23
  • Thanks. But is there still a problem with locking in virtual network when creating several VMs in succession? I was understanding that some of jobs didn't complete successfully because of the exclusive lock. Did you modify your script to handle failed jobs until they all completed? – Glenn Nov 05 '14 at 22:04
1

Have you tried moving the Wait-job to outside of the for loop? I'm guessing you know that by putting it there you are making it run synchronously.

The following will wait for all jobs:

Get-Job | Wait-Job
Get-Job | Receive-Job

The Receive-Job should give you some clues about why some are not being created.

David Martin
  • 11,764
  • 1
  • 61
  • 74