Terraform Plan Fails in Azure Devops

Question

I am trying to use terraform to build the infra in Azure. I am automating it through Azure DevOps, we don't have the tasks in our org yet, I am running through CLI scripts to get it, while I am able to till terraform init but unable to run through terraform plan. I am using service principal to authenticate as mentioned here. I am following this to complete the setup.

Here is my pipeline.

- task: AzureCLI@1
  displayName: Terraform init
  inputs:
    azureSubscription: Subscription
    scriptLocation: inlineScript
    inlineScript: |
      set -eux  # fail on error
      terraform init \
        -backend-config=storage_account_name=$(storageAccountName) \
        -backend-config=container_name=$(container_name) \
        -backend-config=key=$(key)/terraform.tfstate \
        -backend-config=sas_token=$(artifactsLocationSasToken) \
        -backend-config=subscription_id="$(ARM_SUBSCRIPTION_ID)" \
        -backend-config=tenant_id="$(ARM_TENANT_ID)" \
        -backend-config=client_id="$(ARM_CLIENT_ID)" \
        -backend-config=client_secret="$(ARM_CLIENT_SECRET)" 
    addSpnToEnvironment: true
    workingDirectory: $(System.DefaultWorkingDirectory)/Modules

- bash: |
    set -eu  # fail on error
    terraform plan -out=tfplan -input=false -detailed-exitcode
   
  displayName: Terraform apply
  workingDirectory: $(System.DefaultWorkingDirectory)/Modules

and in the tf file I have very basic to try

provider "azurerm" {
  version     = ">= 2.61.0"
  features {}
}

data "azurerm_resource_group" "main" {
  name = var.resource_group_name
}

terraform {
  backend "azurerm" {
 }
}

I am getting this error.

Error building AzureRM Client: obtain subscription(***) from Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1: ERROR: Please run 'az login' to setup account.

---

Updated

- task: AzureCLI@2
  inputs:
    azureSubscription: $(scConn)
    scriptType: 'pscore'
    scriptLocation: 'inlineScript'
    inlineScript: |
      $sasToken = (az storage container generate-sas --account-name $(storageAccountName) --name $(container_name) --permissions rwdl --expiry $(date -u -d "30 minutes" +%Y-%m-%dT%H:%MZ))
      Write-Host($sasToken) Write-Output("##vso[task.setvariable variable=artifactsLocationSasToken;]$sasToken")

- task: AzureCLI@1
  displayName: Terraform credentials
  inputs:
    azureSubscription: $(scConn)
    scriptLocation: inlineScript
    inlineScript: |
      set -eu  # fail on error
      echo "##vso[task.setvariable variable=ARM_CLIENT_ID]$(servicePrincipalId)"
      echo "##vso[task.setvariable variable=ARM_CLIENT_SECRET;issecret=true]$(servicePrincipalKey)"
      echo "##vso[task.setvariable variable=ARM_SUBSCRIPTION_ID]$(subscriptionId)"
      echo "##vso[task.setvariable variable=ARM_TENANT_ID]$(tenantId)"
    addSpnToEnvironment: true

- task: AzureCLI@1
  displayName: Terraform init
  inputs:
    azureSubscription: $(scConn)
    scriptLocation: inlineScript
    inlineScript: |
      set -eux  # fail on error
      terraform init \
        -backend-config=storage_account_name=$(storageAccountName) \
        -backend-config=container_name=$(container_name) \
        -backend-config=key=$(key)/terraform.tfstate \
        -backend-config=sas_token=$(artifactsLocationSasToken) \
        -backend-config=subscription_id="$(ARM_SUBSCRIPTION_ID)" \
        -backend-config=tenant_id="$(ARM_TENANT_ID)" \
        -backend-config=client_id="$(ARM_CLIENT_ID)" \
        -backend-config=client_secret="$(ARM_CLIENT_SECRET)" 
    addSpnToEnvironment: true
    workingDirectory: $(System.DefaultWorkingDirectory)/Modules

Answer 1

Error building AzureRM Client: obtain subscription(***) from Azure CLI: Error parsing json result from the Azure CLI: Error waiting for the Azure CLI: exit status 1: ERROR: Please run 'az login' to setup account.

The root cause of this issue is that Azure CLI will run the az account clear command at the end. So the az login information in the current Azure CLI task will not be retained.

You need to add additional command (az login command)to login before the terraform plan command.

You could enable the parameter in Azure CLI Task: addSpnToEnvironment: true and set the login info as Pipeline variable. Then you could use these info in az login command.

Here is an example:

- task: AzureCLI@1
  displayName: Terraform init
  inputs:
    azureSubscription: Subscription
    scriptType: bash
    scriptLocation: inlineScript
    inlineScript: |
      set -eux  # fail on error
     echo "##vso[task.setvariable variable=ARM_CLIENT_ID]$servicePrincipalId" 
     echo "##vso[task.setvariable variable=ARM_CLIENT_SECRET]$servicePrincipalKey"
     echo "##vso[task.setvariable variable=ARM_TENANT_ID]$tenantId"

      terraform init \
        -backend-config=storage_account_name=$(storageAccountName) \
        -backend-config=container_name=$(container_name) \
        -backend-config=key=$(key)/terraform.tfstate \
        -backend-config=sas_token=$(artifactsLocationSasToken) \
        -backend-config=subscription_id="$(ARM_SUBSCRIPTION_ID)" \
        -backend-config=tenant_id="$(ARM_TENANT_ID)" \
        -backend-config=client_id="$(ARM_CLIENT_ID)" \
        -backend-config=client_secret="$(ARM_CLIENT_SECRET)" 
    addSpnToEnvironment: true
    workingDirectory: $(System.DefaultWorkingDirectory)/Modules

- bash: |
    set -eu  # fail on error
    az login --service-principal --username $(ARM_CLIENT_ID) --password $(ARM_CLIENT_SECRET)  --tenant $(ARM_TENANT_ID)
    terraform plan -out=tfplan -input=false -detailed-exitcode
   
  displayName: Terraform apply
  workingDirectory: $(System.DefaultWorkingDirectory)/Modules

Answer 2

This is a very late update. I used different tasks instead of the figuring out everything subsequently using the cli tasks.

  - task: ms-devlabs.custom-terraform-tasks.custom-terraform-installer-task.TerraformInstaller@0
    displayName: 'Install Terraform'
    inputs:
      terraformVersion: version

  - task: TerraformTaskV2@2
    inputs:
      provider: 'azurerm'
      command: 'init'
      workingDirectory: $(Pipeline.Workspace)/${{ parameters.workingDir }}
      backendServiceArm: ${{ parameters.armSC }}
      backendAzureRmResourceGroupName: $(rg_name)
      backendAzureRmStorageAccountName: $(str_name)
      backendAzureRmContainerName: $(str_tf_containername)
      backendAzureRmKey: $(common_statefile)

  - task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV2@2
    displayName: 'Terraform : azurerm plan'
    inputs:
      command: plan
      workingDirectory: $(Pipeline.Workspace)/${{ parameters.workingDir }}
      commandOptions: '-out=tfplan'
      environmentServiceNameAzureRM: ${{ parameters.armSC }}
      backendServiceArm: ${{ parameters.armSC }}
      backendAzureRmResourceGroupName: $(rg_name)
      backendAzureRmStorageAccountName: $(str_name)
      backendAzureRmContainerName: $(str_tf_containername)
      backendAzureRmKey: $(common_statefile)

  - task: ms-devlabs.custom-terraform-tasks.custom-terraform-release-task.TerraformTaskV2@2
    displayName: 'Terraform : azurerm apply'
    name: TerraformOutputs
    inputs:
      command: apply
      workingDirectory: $(Pipeline.Workspace)/${{ parameters.workingDir }}
      commandOptions: '-auto-approve tfplan'
      environmentServiceNameAzureRM: ${{ parameters.armSC }}
      backendServiceArm: ${{ parameters.armSC }}
      backendAzureRmResourceGroupName: $(rg_name)
      backendAzureRmStorageAccountName: $(str_name)
      backendAzureRmContainerName: $(str_tf_containername)
      backendAzureRmKey: $(common_statefile)