0

I can see that with Custom Script Extension it is possible to bootstrap new VMs (in Scale Set). To access a script it needs azure storage URI and credentials. This approach doesn't work for me because (internal policies) it's not allowed to pass storage credentials.

My VMSS has assigned service identity, the latter is registered with KeyVault. So, it is quite straightforward to get credentials directly on a box. But for this I need at least small bootstrap script =)

I found one hacky way how to achieve this through Custom Script Extension:

$bootstrapScriptPath = Join-Path -Path $PSScriptRoot -ChildPath "bootstrap.ps1"
$bootstrapScriptBlock = get-command $bootstrapScriptPath | Select -ExpandProperty ScriptBlock
$installScriptBase64 = [System.Convert]::ToBase64String([System.Text.Encoding]::Unicode.GetBytes($bootstrapScriptBlock.ToString()))

"commandToExecute": "[concat('powershell -ExecutionPolicy Unrestricted -EncodedCommand ', parameters('installScriptBase64'))]"

But I wonder whether there are better solutions.

Essentially I need something which Cloud Service provides - ability to upload payload and config settings.

SOLUTION

(note, this is for Windows VM. For Linux VM there is an easier way - thanks to @sendmarsh)

Please see below for actual implementation (note, I marked as answer a post from @4c74356b41 who suggested this idea).

ZakiMa
  • 5,637
  • 1
  • 24
  • 48

3 Answers3

1

First of all, I dont really see anything hacky, its a valid approach.

Another way to pass in data - using custom data property. It will be available as a file inside vm, I dont remember if its base64 encoded, but you can quickly figure it after provisioning.

Yet another approach is to use Managed Service Identity for the VM. that way you just assign VM proper permissions and it can download script from the storage without you passing them in explicitly.

Either way, you need to pass in your script to the vm and invoke it using a script extension. you can use custom image with script inside it. or you can have the script in the publicly available url so vm can always pull it and execute it (you need MSI to handle auth for you in this case).

Another thing you can do is pull certificates from KV directly inside the VM during. provisioning.

4c74356b41
  • 69,186
  • 6
  • 100
  • 141
  • Thank you for a hint with custom data. I'll try approach with passing bootstrap script through Custom Script Extension and passing config settings through custom data (which keyvault, which user identity, etc). This should be sufficient to securely download the whole payload. – ZakiMa Feb 15 '19 at 08:07
  • 1
    Solution worked, thank you! Provided actual implementation as well. – ZakiMa Feb 22 '19 at 07:56
1

You could avoid using a script extension with Custom Data and cloud-init - if it's a Linux VM. Not using a script extension will save you a couple of minutes from your deployment time too.

There's an example for a VM here: https://msftstack.wordpress.com/2018/11/26/speeding-up-azure-resource-manager-templates-using-cloud-init/ - you can follow the same method for a scale set.

sendmarsh
  • 1,046
  • 7
  • 11
  • It's great that such option exists for Linux VM, upvoted! I need to run on Windows VM (otherwise it would be AKS rather than VMSS). But i think your answer will help people coming here for Linux VMs. Thank you! – ZakiMa Feb 15 '19 at 08:05
  • totally not copying my answer ;) – 4c74356b41 Feb 22 '19 at 07:41
  • Your answer was good. Not copying though. The reason for the separate answer is that your answer stated "Either way, you need to pass in your script to the vm and invoke it using a script extension". My answer was clarifying that script extension is not necessary on Linux. – sendmarsh Feb 24 '19 at 16:19
1

Here is where I ended up:

  1. Pass Bootstrap.ps1 as custom data
  2. Run a complicated command through Custom Script Extension - it decodes CustomData.bin, copies as Bootstrap.ps1 and calls it with parameters
  3. No external files are downloaded
  4. Bootstrap.ps1 registers a task scheduler to rerun, gets a token from metadata service for user assigned managed identity, goes to Key Vault to get credentials to download main payload, downloads it, unzips, etc.

Below snippets for how to make #1 & #2 work.

Sample Bootstrap.ps1:

param(
 [Parameter(Mandatory=$True)]
 [string]
 $param1,

 [Parameter(Mandatory=$True)]
 [string]
 $param2
)

$ErrorActionPreference = "Stop"

Write-Output("Running Bootstrap.ps1 with the following parameters:");
Write-Output("`$param1 = `"$param1`";");
Write-Output("`$param2 = `"$param2`";");

Pass it as CustomData:

# Encoding bootstrap script
$bootstrapScriptPath = (Join-Path -Path "." -ChildPath "Node/Bootstrap.ps1");
$bootstrapScriptBlock = get-command $bootstrapScriptPath | Select -ExpandProperty ScriptBlock;
$encodedScript = [System.Convert]::ToBase64String([System.Text.Encoding]::Unicode.GetBytes($bootstrapScriptBlock.ToString()));
...
| Set-AzVMOperatingSystem -CustomData $encodedScript

Command line:

$commandLine = "powershell -ExecutionPolicy Unrestricted -Command `"" ` + # Running powershell in cmd
                    "`$ErrorActionPreference = 'Stop';" ` + # Upon any error fail the provisioning of the extension
                    "`$content = [IO.File]::ReadAllText('C:\AzureData\CustomData.bin');" + ` # Read Base64-encoded file
                    "`$bytes = [System.Convert]::FromBase64String(`$content);" + ` # Convert to bytes
                    "`$script = [System.Text.Encoding]::Unicode.GetString(`$bytes);" + ` # Decode to string
                    "[IO.File]::WriteAllText('C:\AzureData\Bootstrap.ps1', `$script);" + ` # Save as Bootstrap.ps1
                    "C:\AzureData\Bootstrap.ps1 " + ` # Run a script
                    "-param1 'test' -param2 'test' " + ` # Pass needed parameters
                    " | Out-File -PSPath 'C:\AzureData\output.log';" ` +
                "`"";

Custom Script Extension:

$extensionSettings = @{ "fileUris" = ""; "commandToExecute" = ""};
$extensionProtectedSettings = @{ "commandToExecute" = "$commandLine" };
$result = Set-AzVMExtension -VMName "$($vm.Name)" -Location $resourceGroupLocation -Publisher "Microsoft.Compute" -Type "CustomScriptExtension" `
                            -TypeHandlerVersion "1.9" -Settings $extensionSettings -ProtectedSettings $extensionProtectedSettings `
                            -Name "Bootstrap" -ResourceGroupName $resourceGroup.ResourceGroupName;
ZakiMa
  • 5,637
  • 1
  • 24
  • 48