Properly using Terraform External Data Source

Question

I am using Terraform from the bash cloud shell in Azure. I am trying to add an external data source to my Terraform configuration file that will use az cli to query for the virtualip object on a Microsoft.Web/hostingEnvironment the template deploys.

AZ CLI command line: az resource show --ids /subscriptions/<subscription Id>/resourceGroups/my-ilbase-rg/providers/Microsoft.Web/hos tingEnvironments/my-ilbase/capacities/virtualip

Output when run from command line:

   {
      "additionalProperties": {
        "internalIpAddress": "10.10.1.11",
        "outboundIpAddresses": [
          "52.224.70.119"
        ],
        "serviceIpAddress": "52.224.70.119",
        "vipMappings": []
      },
      "id": null,
      "identity": null,
      "kind": null,
      "location": null,
      "managedBy": null,
      "name": null,
      "plan": null,
      "properties": null,
      "sku": null,
      "tags": null,
      "type": null
    }

In my Terraform config I create a variable for the --ids value:

    variable ilbase_resourceId {
      default = "/subscriptions/<subscription Id>/resourceGroups/my-ilbase-rg/providers/Microsoft.Web/hostingEnvironments/my-ilbase/capacities/virtualip"
    }

I then have the data source structured this way:

    data "external" "aseVip" {
      program = ["az", "resource", "show", "--ids", "${var.ilbase_resourceId}"]
    }

When I execute my configuration, I get the error below:

    data.external.aseVip: data.external.aseVip: command "az" produced invalid JSON: json: cannot unmarshal object into Go value of type string

Any ideas what I am doing wrong?

It's most likely the login. terraform doesn't run commands in a shell, so the token used when you run AZ in the shell is not available when Terraform runs it — Simon Halsey, Mar 24 '18 at 16:32

score 2 · Accepted Answer · answered Mar 25 '18 at 12:34

I discovered the problem was that the Terraform External Data Source is not yet able to handle the complex structure of what gets returned by the command. I was able to get around this by adding an AZ CLI command block at the beginning of the script I use to deploy the Application Gateway that grabs the IP address and passes it into the Terraform config as a variable. Below is the script block I am using:

ilbase_virtual_ip=$(
  az resource show \
  --ids "/subscriptions/$subscription_id/resourceGroups/$ilbase_rg_name/providers/Microsoft.Web/hostingEnvironments/$ilbase_name/capacities/virtualip" \
  --query "additionalProperties.internalIpAddress"
)

score 0 · Answer 2 · answered Mar 10 '18 at 17:21

0

That command will be successful when you are working in a session. I guess that when you run it from your shell, you already have done az login. When terraform executes your command, it is not using your existing session. You would need to create a PS1 script where you would be propmted for login, or where you provide your credentials so your request can be successful.

Whatever your choice is, take into account that the ONLY output that script should have is a JSON. If any other command add something to the output (for example, when you do a login, you have an output with information about your subscription) then you will have the same error as the output is not a proper JSON. You will need to pipeline that kind of extra ouputs to Out-Null making them "silent" and just write to the output the JSON you are receiving from your request.

I hope this can help.

answered Mar 10 '18 at 17:21

Raul Vega

121
2

Should be no need to authenticate in Cloud Shell...you had to authenticate to get into it. Terraform is native to Cloud Shell as is AZ CLI. – phydeauxman Mar 11 '18 at 18:09
Oh! Cloud shell, sorry, my bad. Then if the login is solved, after executing your command in the shell is the JSON the only output? That was my issue when I tried to use external data source. – Raul Vega Mar 12 '18 at 09:04
Yes, if I manually execute the az cli command from within the cloud shell, it returns the JSON formatted info out I listed in the original post. – phydeauxman Mar 12 '18 at 11:39

score 0 · Answer 3 · answered Aug 02 '22 at 23:28

While there is an accepted response, that is actually a good workaround. The error is because terraform expect an one level json map like { "a" = "b", "c" = "d" } and your az command returns a multi level map. ( a map of maps )

You can improve your az command to limit the return only one map by adding --query

data "external" "aseVip" {
      program = ["az", "resource", "show", "--ids", "${var.ilbase_resourceId}" , "--query additionalProperties" ]
    }

output "internalIpAddress" {
value = data.external.aseVip.internalIpAddress
}

output "outboundIpAddresses" {
value = data.external.aseVip.outboundIpAddresses
}

I hope this may help other people.

Properly using Terraform External Data Source

3 Answers3