15

I have been using the below to successfully create a back-end state file for terraform in Azure storage, but for some reason its stopped working. I've recycled passwords for the storage, trying both keys and get the same error every-time

backend.tf

    terraform {
    backend "azurerm" {
        storage_account_name    = "terraformstorage"
        resource_group_name     = "automation"
        container_name          = "terraform" 
        key                     = "testautomation.terraform.tfstate"
        access_key              = "<storage key>"
    }
}

Error returned

terraform init
Initializing the backend...
Successfully configured the backend "azurerm"! Terraform will automatically use this backend unless the backend configuration changes.
Error refreshing state: storage: service returned error: StatusCode=403, ErrorCode=AuthenticationFailed, ErrorMessage=Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:665e0067-b01e-007a-6084-97da67000000
Time:2018-12-19T10:18:18.7148241Z, RequestInitiated=Wed, 19 Dec 2018 10:18:18 GMT, RequestId=665e0067-b01e-007a-6084-97da67000000, API Version=, QueryParameterName=, QueryParameterValue=

Any ideas what im doing wrong?

Dijkgraaf
  • 11,049
  • 17
  • 42
  • 54
Staggerlee011
  • 847
  • 2
  • 13
  • 23
  • It looks like your credentials are wrong. Can you try accessing the state file in the blob storage via another mechanism? Does Azure have a useful CLI tool you can use to test this? – ydaetskcoR Dec 19 '18 at 13:44
  • hi ydaetskcoR, I can connect successfully to the storage using "Azure Storage Explorer" using storage account name and key, which is pretty much all that terraform uses. just to add to the confusion! – Staggerlee011 Dec 19 '18 at 15:38
  • Another option to explore: make sure the container_name match the container used to generate the sas_token – Lolorol Jun 06 '21 at 14:59
  • I was deploying from Azure DevOps pipeline and created a service connection from the UI. I need to very the service connection again and enter my credentials. After that it works again. – Playing With BI Aug 18 '22 at 13:55

6 Answers6

18

What worked for me is to delete the local .terraform folder and try again.

vgaltes
  • 1,150
  • 11
  • 18
12

Another problem can be time resolution.

I experienced those problems as well, tried all the above mentioned steps, but nothing helped.

What happened on my system (Windows 10, WSL2) was, that WSL lost its time sync and I was hours apart. This behaviour is described in https://github.com/microsoft/WSL/issues/4245.

For me it helped to

  • get the appropriate time in WSL (sudo hwclock -s) and

  • to reboot WSL

Hope, this will help others too.

MaxiPalle
  • 410
  • 1
  • 6
  • 15
  • 2
    This worked for me too. I only had to run hwclock -s, didn't need to reboot. As soon as I saw the command I just had to roll my eyes. This was causing a problem in Apache Airflow for me too (solved by running the same command) - the cause, when my laptop goes to sleep, WSL takes a nap too, so looses time. I may just create a script in my bash profile as a temporary fix. – Coffee and Code Oct 14 '20 at 02:51
  • 1
    you can also run `wsl -d docker-desktop hwclock -s` in the host. – Hannes Nel Dec 09 '20 at 21:19
  • Thanks @MaxiPalle, that resolved it for me too. So many issues can be caused by the date sync issue. The worst part is it's intermittent, so I forgot about it. I appended `sudo hwclock -s` to my .profile file, just so I don't forget about this issue, but this terminal had been running overnight so I forgot. – Coffee and Code Jan 25 '21 at 23:00
8

Here are few suggestions:

  • Run: terraform init -reconfigure.
  • Confirm your "terraform/backend" credentials.
  • In case your Terraform contains some "azurerm_storage_account/network_rules" to allow certain IP addresses, or make sure you're connected to the right VPN network.
  • If above won't work, run TF_LOG=TRACE terraform init to debug further.
kenorb
  • 155,785
  • 88
  • 678
  • 743
2

Please ensure you've been authenticated properly to Azure Cloud.

If you're running Terraform externally, re-run: az login.

If you're running Terraform on the instance, you can use managed identities, or by defining the following environmental variables:

ARM_USE_MSI=true
ARM_SUBSCRIPTION_ID=xxx-yyy-zzz
ARM_TENANT_ID=xxx-yyy-zzz

or just run az login --identity, then assign the right role (azurerm_role_assignment, e.g. "Contributor") and appropriate policies (azurerm_policy_definition).

See also:

kenorb
  • 155,785
  • 88
  • 678
  • 743
0

I was facing the same issue while setting the remote backend state to Azure StorageV2. I was using SAS token to set the remote backend state. The token was generated using Terraform data provider (data "azurerm_storage_account_sas" ""). The SAS Token thus generated, was not working. So, I had to manually generate SAS token from Azure portal. That fixed the problem.

Prem
  • 303
  • 2
  • 9
-2

There should a .terraform directory , where you are running the terraform init command from. Remove .terraform or move it to Someotehr name. Next time terraform init runs , it will recreate that directory with new init.

JayTee
  • 1