Unable to clone (download) terraform modules from Azure repos using Azure pipelines

Question

I am not able to download Terraform modules located at Azure repos.

Azure build pipeline yaml file is like the below:

trigger:
- feature/iac_create

pool:
  vmImage: ubuntu-latest

steps:
- script: |
    pwd  
    ls -larth
    cd infra/
  displayName: "From where is script running"

- task: TerraformTaskV4@4
  inputs:
    provider: 'azurerm'
    command: 'init'
    backendServiceArm: 'Development(xxxxxxxxxxxxxxxxxxxxxx)'
    backendAzureRmResourceGroupName: 'terraform-state-files'
    backendAzureRmStorageAccountName: 'sttfstateinterfloradev'
    backendAzureRmContainerName: 'dev-commercetoolstod365handler'
    backendAzureRmKey: 'dev.tfstate'
    workingDirectory: '$(System.DefaultWorkingDirectory)/infra'
  # env:
  #   ARM_ACCESS_KEY: $(AZURE_PAT)
    
- task: TerraformTaskV4@4
  inputs:
    provider: 'azurerm'
    command: 'plan'
    commandOptions: '-var-file=dev.tfvars out=tfplan'
    workingDirectory: '$(System.DefaultWorkingDirectory)/infra'

- task: TerraformTaskV4@4
  inputs:
    provider: 'azurerm'
    command: 'apply'
    commandOptions: '-var-file=dev.tfvars tfplan'
    workingDirectory: '$(System.DefaultWorkingDirectory)/infra'

The main.tf file is below: [Note: This file + many other tf files are at infra/ folder. ]

module "resource_group" {
  source           = "git::https://'ameya.agashe@'longstringpattokenishere'@dev.azure.com/interfloraau/Spring/_git/spring-terraform-azurerm-resourcegroup?ref=0.2.0"
  location         = var.location
  application_name = var.application_name
  environment_name = var.environment_name
  department       = var.department
  cost_centre      = var.cost_centre
}

When I want to clone the Repo, the RG tef module URL is

https://interfloraau@dev.azure.com/interfloraau/Spring/_git/spring-terraform-azurerm-resourcegroup

The later ?ref=0.2.0 is a tag I want to download.

Unfortunately, I am getting errors like below:

│ Error: Failed to download module
│ 
│ Could not download module "resource_group" (main.tf:1) source code from
│ "git::https://'ameya.agash@***'@dev.azure.com/interfloraau/Spring/_git/spring-terraform-azurerm-resourcegroup?ref=0.2.0":
│ error downloading
│ 'https://%27ameya.agash%40***%27@dev.azure.com/interfloraau/Spring/_git/spring-terraform-azurerm-resourcegroup?ref=0.2.0':
│ /usr/bin/git exited with 128: Cloning into
│ '.terraform/modules/resource_group'...
│ fatal: could not read Password for
│ 'https://'ameya.agashe@***'@dev.azure.com':
│ terminal prompts disabled
│ 
╵

╷
│ Error: Failed to download module
│ 
│ Could not download module "resource_group" (main.tf:1) source code from
│ "git::https://'ameya.agash@***'@dev.azure.com/interfloraau/Spring/_git/spring-terraform-azurerm-resourcegroup?ref=0.2.0":
│ error downloading
│ 'https://%27ameya.agash%40***%27@dev.azure.com/interfloraau/Spring/_git/spring-terraform-azurerm-resourcegroup?ref=0.2.0':
│ /usr/bin/git exited with 128: Cloning into
│ '.terraform/modules/resource_group'...
│ fatal: could not read Password for
│ 'https://'ameya.agash@***'@dev.azure.com':
│ terminal prompts disabled

I am using PAT_TOKEN, which I created with Azure DevOps with code read and write permissions.

As you see, I did try with the environment variable as well, but I got the same error, unable to read from the terminal error.

P.S: I know SSH way, but the instructions will work between My laptop and Azure repos; that is not the problem.

I can write TF modules from my Mac to Azure repos, which work seamlessly.

I want the Azure pipeline to be able to download TF modules and be able to create infrastructure.

Answer 1

Ok, so when you want to use Azure repos for terraform modules, you need to use Azure PAT Token after https WITHOUT any single or double quotes or even using a username.

Therefore the correct syntax is:

module "resource_group" {
  source           = "git::https://<PAT_TOKEN>@dev.azure.com/interfloraau/Spring/_git/spring-terraform-azurerm-resourcegroup?ref=0.2.0"
  location         = var.location
  application_name = var.application_name
  environment_name = var.environment_name
  department       = var.department
  cost_centre      = var.cost_centre
}

I am trying to figure out how to hide that plain text and have it passed as a parameter.

Just as an FYI and not relevant but that PAT TOKEN has been invalidated :)