GitLab Managed Terraform tf.state backend fails

Question

I am trying to run Terraform with AWS provider using GitLab CI/CD. I was using the backend as S3 to store the tf.state file and everything was working fine.

As part of a research, I wanted to experiment with GitLab managed Terraform tf.state. The documentation can be found in: https://gitlab.mvtest-harbor.live/help/user/infrastructure/iac/terraform_state.md

I followed the same procedure as the document said. This is how my backend file looks like:

backend.tf

data "terraform_remote_state" "gitlab-terraform-remote-state" {
  backend = "http"

  config = {
    address = "https://XXXXXX.com/api/v4/projects/32/terraform/state/gitlab-terraform.tfstate"
    username = "arjun"
    password = "password"
  }
}

Also I added the below as environment variables in CI/CD in GitLab server.

PROJECT_ID="" TF_USERNAME=""
TF_PASSWORD=""
TF_ADDRESS="https://gitlab.com/api/v4/projects/${PROJECT_ID}/terraform/state/old-state-name"

In the .gitlab.ci.yaml, this is how I had described my job:

.gitlab-ci.yaml

terraform plan:
  stage: terraform-plan
  dependencies:
    - lambda package build
  variables:
    PLAN: plan.tfplan
    JSON_PLAN_FILE: tfplan.json
    STATE: dbrest.tfstate
  cache:
    paths:
      - .terraform
  before_script:
    - alias convert_report="jq -r '([.resource_changes[]?.change.actions?]|flatten)|{\"create\":(map(select(.==\"create\"))|length),\"update\":(map(select(.==\"update\"))|length),\"delete\":(map(select(.==\"delete\"))|length)}'"
    - cd terraform
    - terraform --version
    - terraform init -backend-config=address=${TF_ADDRESS} -backend-config=lock_address=${TF_ADDRESS}/lock -backend-config=unlock_address=${TF_ADDRESS}/lock -backend-config=username=${TF_USERNAME} -backend-config=password=${TF_PASSWORD} -backend-config=lock_method=POST -backend-config=unlock_method=DELETE -backend-config=retry_wait_min=5
  script:
    - cp ../artifacts/$CI_PIPELINE_ID.zip ./
    - terraform plan -out=plan_file
    - terraform show --json plan_file > plan.json
  artifacts:
    paths:
      - plan.json
    expire_in: 2 weeks
    when: on_success
    reports:
      terraform: plan.json
  only:
    - main
  allow_failure: false
  needs: ['lambda package build']

Once the pipeline runs, after executing the terraform init from the before_script, I am getting this error:

│ Error: Unable to find remote state
│
│ with data.terraform_remote_state.gitlab-terraform-remote-state,
│ on backend.tf line 1, in data "terraform_remote_state" "gitlab-terraform-remote-state":
│ 1: data "terraform_remote_state" "gitlab-terraform-remote-state" {
│
│ No stored state was found for the given workspace in the given backend.

Does anyone know why this is happening?
Does anyone know how to fix this?

GitLab Managed Terraform tf.state backend fails

0 Answers0