1

I want to utilize MSI e.g. to access KeyVault in applications (in particular for me: Azure Functions runtime) hosted inside containers which are hosted in a Azure Service Fabric VMSS.

What do I need to do to achieve this?

Kai Walter
  • 3,485
  • 2
  • 32
  • 62

1 Answers1

6

based on a hint on this issue:

Step 1 - add identity to VMSS

Extend ARM template for clusters Microsoft.Compute/virtualMachineScaleSets resource. Add identity element on the root level of the resource , like with properties 

...
  "identity": {
    "type": "SystemAssigned"
  },      
...

(Re-)deploy the cluster.

Step 2 - add routing to containers

In Windows containers the routing to the MSI endpoint is not working by default. For that I added an entry script e.g. Entry.PS1 (do not forget to add the original ENTRYPOINT of your container - ServiceMonitor.exe in my case because I have an IIS container):

Write-Host "adding route for Managed Service Identity"
$gateway = (Get-NetRoute | Where-Object {$_.DestinationPrefix -eq '0.0.0.0/0'}).NextHop
$arguments = 'add','169.254.169.0','mask','255.255.255.0',$gateway
&'route' $arguments

$response = Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fvault.azure.net%2F' -Method GET -Headers @{Metadata="true"} -UseBasicParsing
Write-Host "MSI StatusCode :" $response.StatusCode

C:\ServiceMonitor.exe w3svc

and modified the Dockerfile / containers ENTRY:

...
ENTRYPOINT ["powershell.exe","C:\\entry.PS1"]

Background: adding the route add not at entry point level will execute the statement at build time and add the route to the build host/container

Step 3 - optional re-image VMSS nodes

However I still experienced a problem with an existing cluster. When acessing the token endpoint with

Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fvault.azure.net%2F' -Method GET -Headers @{Metadata="true"} -UseBasicParsing

I still got this error

Invoke-WebRequest : Unable to connect to the remote server
At line:1 char:1
+ Invoke-WebRequest -Uri 'http://169.254.169.254/metadata/identity/oaut ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

to fix this I had to re-image the VMSS nodes

Kai Walter
  • 3,485
  • 2
  • 32
  • 62
  • so you are saying containers inside vmss nodes (vms) have same level of permissions as the parent vmss\vm got? well, as long as they can talk to the management endpoint – 4c74356b41 Sep 30 '18 at 13:12
  • correct - no container but host/VM specific authorization – Kai Walter Sep 30 '18 at 16:57
  • For images that don't have powershell, I'm creating a batch file with these two lines instead: for /f "tokens=3" %%a in ('route print 0.0.0.0 -4 ^| findstr /c:"0.0.0.0"') do route add 169.254.169.0 mask 255.255.255.0 %%a if %errorlevel% == 0 %1 %2 %3 %4 %5 %6 %7 %8 %9 And the docker file: ENTRYPOINT ["cmd", "/c", "LaunchWithMSI.bat", "dotnet", "MyCoolApp.dll"] (Still requires running the container app as admin though) – Josh Oct 31 '18 at 23:56
  • @KaiWalter is the entry script something I could run on startup of a VM so all the containers can then fetch the endpoint and I don't have to modify docker files? My use case is slightly different, I am just interested in using Managed Identities for docker containers on an Azure VM, until I move the services into a more Cloud Native world. – pijemcolu Apr 20 '21 at 15:28
  • 1
    @pijemcolu - yes I would guess this will work also on Docker on Azure VM. The principles and underlying infrastructure are the same. – Kai Walter Apr 21 '21 at 04:41