How can a script access Service Connections? (Azure Devops Pipelines)

Question

According to https://learn.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints there's a rich array of Service Connection types. I can easily manage a set of service connections at the project level and set permissions to limit which users are able to view/edit them -- this is all good.

But I can't figure out how to access a Service Connection with a script step in my build pipeline. For example, let's say I have a Service Connection representing credentials for an Azure Service Principal. I'd like to access those credentials in a script step.

How can I write a script step that makes use of them?

Answer 1

Because a Service Connection involves data shaped specifically to the connected service (the Generic Service Connection being the exception that proves the rule...), you won't be able to make use of strongly typed properties in your Bash task. Instead, you may want to examine environment variables and process the service connection data manually.

Based on a survey of some of the tasks in the Azure DevOps repos, it appears that service connections and their data are populated as environment variables on the agent running the build task. The service connections are retrieved via a method that runs a given name string through the following regex before retrieving the resultant environment key's value:

process.env[name.replace(/\./g, '_').toUpperCase()];

The retrieval of various Service Endpoint data is wrapped in the vsts-task-lib/task module, allowing consuming tasks to write code like so:

taskLib.getEndpointAuthorization('SYSTEMVSSCONNECTION', false);

taskLib.getEndpointDataParameter('MYSERVICECONNECTION', 'SOME_PARAMETER_NAME', false);

taskLib.getEndpointUrl('MYSERVICECONNECTION', false) // <-- last param indicates required or not

Therefore, if you wanted to access service connections in a bash script without any additional customization, I would recommend that you:

a) Validate the availability of service connection information in the build script task by iterating and writing environment variables, setting the system.debug environment variable. There's some indication that build tasks aren't "seeded" with connections they aren't requesting specifically, so you may need to create a custom build task which has as one of its' inputs the service connection name you want to use

b) read the desired values from variables as outlined above in your bash script. Service connection variable names may be computed similarly to this:

   var dataParam = getVariable('ENDPOINT_DATA_' + id + '_' + key.toUpperCase());  

You may need to iterate against this to determine the data schema/structure.

Answer 2

I've been wondering about this too. The solution I've settled on is to use the 'Azure CLI' task rather than the basic 'Script' (or 'Bash') task. This is ostensibly for running Az CLI commands, but there's nothing to stop you running only standard Bash scripts (or PSCore if that's your thing).

If you examine the environment variables present when you run this task, you'll see a bunch of information about the Service Connection in variables prefixed with 'ENDPOINT_DATA_'. This tallies up with what Josh E was saying. It includes Azure Subscription ID, name, Service Principle Object ID, etc.

Optionally you can enable the Service Principle details to be added to the environment too. This will then include SPN key, TenantID, etc. as secret environment variables.

Here's what the tasks look like:

- task: AzureCLI@2
  displayName: 'Azure CLI'
  inputs:
    scriptType: bash
    scriptLocation: inlineScript
    azureSubscription: '<Service Connection Name>'
    inlineScript: |
      env | sort

- task: AzureCLI@2
  displayName: 'Azure CLI, with SPN info'
  inputs:
    scriptType: bash
    scriptLocation: inlineScript
    azureSubscription: '<Service Connection Name>'
    addSpnToEnvironment: true
    inlineScript: |
      env | sort

Of course this is all only applicable to Azure Cloud Service Connections. There might be similar techniques you could use for other Service Connections, but I haven't investigated them.

Answer 3

I found that if I use the Kubectl task with the command to login right before I run my bash Task, I do not need to authenticate or use a hostname.

KUBERNETESNODE and SERVICEPROTOCOL are Pipeline variables that I set a priori.

      - task: Kubernetes@1
        displayName: 'Kubernetes Login'
        # This is needed to run kubectl command from bash.
        inputs:
          connectionType: 'Kubernetes Service Connection'
          kubernetesServiceEndpoint: '<Service Connection Name>'
          command: 'login'

      - task: Bash@3
        displayName: 'Run Component Test'        
        inputs:
          targetType: 'inline'
          script: |
            #Get the Node Port
            nodePort=`kubectl get --namespace $(Build.BuildId) svc <service name> -o=jsonpath='{.spec.ports[0].nodePort}'`
            #Run Newman test
            newman run postman/Service.postman_collection.json --global-var host=$KUBERNETESNODE --global-var protocol=$SERVICEPROTOCOL --global-var port=$nodePort -r junit

Answer 4

I am using the same service connection in my scripts/tools as for the ARM deployments.

In order to export the variables, I created the following template.

parameters:
- name: azureSubscription
  type: string
- name: exportAsOutput
  type: boolean
  default: false
  
steps:  
- task: AzureCLI@2
  name: exported_azure_credentials
  displayName: 'Export Azure Credentials'
  inputs:
    azureSubscription: '${{ parameters.azureSubscription }}'
    scriptType: pscore
    scriptLocation: inlineScript
    addSpnToEnvironment: true
    ${{ if eq(parameters.exportAsOutput, true) }}:
      inlineScript: |
        Write-Host "##vso[task.setvariable variable=AZURE_TENANT_ID]$env:tenantId"
        Write-Host "##vso[task.setvariable variable=AZURE_TENANT_ID;isOutput=true]$env:tenantId"
        Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_ID]$env:servicePrincipalId"
        Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_ID;isOutput=true]$env:servicePrincipalId"
        Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_SECRET]$env:servicePrincipalKey"
        Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_SECRET;isOutput=true]$env:servicePrincipalKey"
    ${{ if eq(parameters.exportAsOutput, false) }}:
      inlineScript: |
        Write-Host "##vso[task.setvariable variable=AZURE_TENANT_ID]$env:tenantId"
        Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_ID]$env:servicePrincipalId"
        Write-Host "##vso[task.setvariable variable=AZURE_CLIENT_SECRET]$env:servicePrincipalKey"

DevOps is really clever about secrets, so they do not show up in the pipeline logs.

Answer 5

As others have stated, there isn't a great built-in way to access Service Connections with a script. As I do not like the workaround of exposing credentials via long-lived environment variables (for security and laziness purposes), I wrote an extension that allows you to utilize a Generic Service Connection with a custom script: https://marketplace.visualstudio.com/items?itemName=cloudpup.authenticated-scripts

It does this by exposing the service connection as environment variables that only last for the lifetime of the single script task:

Service Connection Variable	Environment Variable
url	AS_SC_URL
username	AS_SC_USERNAME
password	AS_SC_PASSWORD

Example

The tasks included in this extension allow you to write succinct pipelines like the following:

steps:
- task: AuthenticatedPowerShell@1  
  inputs:
    serviceConnection: 'Testing Authenticated Shell'
    targetType: inline
    script: 'Write-Host "url: $env:AS_SC_URL | username: $env:AS_SC_USERNAME | password: $env:AS_SC_PASSWORD"'

Answer 6

Yes this can be achieved, I use this method all of the time, you first need a task which will output the credentials into environment variables, you then create your own variables from the environment variables the task outputs, I usually use AzureCLI:

# Set Variables.
- task: AzureCLI@2
  name: setVariables
  displayName: Set Output Variables
  continueOnError: false
  inputs:
    azureSubscription: nameOfAzureServiceConnection
    scriptType: ps
    scriptLocation: inlineScript
    addSpnToEnvironment: true # this must be set to true
    inlineScript: |
      Write-Host "##vso[task.setvariable variable=azureClientId;isOutput=true]$($env:servicePrincipalId)"
      Write-Host "##vso[task.setvariable variable=azureClientSecret;isOutput=true]$($env:servicePrincipalKey)"
      Write-Host "##vso[task.setvariable variable=azureTenantId;isOutput=true]$($env:tenantId)"

Then you can use these new variables you have set in a step below, make sure you call the variables with the task name, so $(taskName.variableName), the below example uses the new variables to set environment variables in a later PowerShell task for Terraform to use for authentication:

- PowerShell: |
     terraform plan -input=false -out=tfplan
 displayName: 'Terraform Plan'
 env:
   ARM_CLIENT_ID: $(setVariables.azureClientId)
   ARM_CLIENT_SECRET: $(setVariables.azureClientSecret)
   ARM_TENANT_ID: $(setVariables.azureTenantId)

ref: https://jimferrari.com/2021/11/15/access-azure-service-connection-via-script/

Answer 7

If you need to use the service connection to get authorized to different services/resources, you can also get the required tokens with the service connection and pass them to scripts that can't use the service connection directly, like:

- task: AzurePowerShell@5
  inputs:
    azureSubscription: 'AzureServiceConnection'
    ScriptType: 'InlineScript'
    Inline: |
      $token = Get-AzAccessToken 
      echo "##vso[task.setvariable variable=accesstoken;]$($token.Token)"
    azurePowerShellVersion: 'LatestVersion'
  
- script: 'echo This is the token: $(accesstoken)'