15

What is the Google Cloud Platform mechanism for locking state file when using Terraform? Something like DynamoDB on AWS...

thanks

Alexander Tarasov
  • 378
  • 3
  • 7
zurekarol
  • 173
  • 2
  • 7

3 Answers3

21

Google Cloud Platform like most of the remote backends natively supports locking. AWS doesn't support locking natively via S3 but it does as you mentioned via DynamoDB.

To run terraform apply, Terraform will automatically acquire a lock; if someone else is already running apply, they will already have the lock, and you will have to wait.

You can run apply with the -lock-timeout=<TIME> parameter to tell Terraform to wait up to TIME for a lock to be released (e.g., -lock-timeout=10m will wait for 10 minutes).

Max Voitko
  • 1,542
  • 1
  • 17
  • 32
  • 2
    This should be the accepted answer, imo, sounds like OP already knows that gcs can be used but is asking about state locking. The doc's for the gcp backed don't make any mention about 'how' to lock state - compared to the s3 backend which specifically mentions dynamodb – iamgeef Apr 03 '20 at 04:57
  • 1
    the key here is that gcs *natively* supports state locking; coming from an AWS background it's almost hard to wrap one's head around this because in other words: GCS does *not* require thinking about state locking because it's handled automagically for us. – Digital Impermanence Dec 27 '22 at 03:04
20

gcs backend implements Terraform state locking by using a special lock file with .tflock extension. This file is placed next to the Terraform state itself for the period of Terraform state operation. For example, if the state file is located at path

gs://BUCKET/PREFIX/WORKSPACE.tfstate

then the corresponding lock file will be located at path

gs://BUCKET/PREFIX/WORKSPACE.tflock

Source: hashicorp/terraform

The atomicity of locking is guaranteed by using the GCS feature called Precondition. Terraform itself makes use of DoesNotExist condition of GCP Go SDK which in turn uses the GCS Precondition. Underneath, this adds this HTTP header x-goog-if-generation-match: 0 to the GCS copy request.

According to GCS documentation:

When a Match precondition uses the value 0 instead of a generation number, the request only succeeds if there are no live objects in the Cloud Storage bucket with the name specified in the request.

Which is exactly what is needed for Terraform state locking.

Alexander Tarasov
  • 378
  • 3
  • 7
  • 2
    I feel like this answer matches what is actually being asked in the question most. Namely, what is the _mechanism_ that state locking is implemented 0in the gcs backend. – Matt Dunn Oct 08 '21 at 13:55
3

Where you store the state files (defined using a backend) is distinct from where you're deploying to. They could be the same, but don't have to be. For example, you could deploy resources to Azure while storing the state file in an AWS S3 bucket.

If you're interested in storing the state file in the Google Cloud, Terraform has a backend called gcs that includes locking. To quote the documentation:

gcs stores the state as an object in a configurable prefix and bucket on Google Cloud Storage (GCS).

KJH
  • 2,382
  • 16
  • 26