I am trying to put together a fairly complex pipeline with several jobs that run sequentially in our different environments. This is to run our Terraform changes across our infra. The sequence of jobs should run automatically across our infraci environment which is only ever rolled out to via CI, then stop and require a button click to start the deployment to our dev environment which has actual (albeit dev) users. Of course I don't want to write the same code over and over again so I've tried to be as DRY as possible. Here is my gitlab-ci.yml:
---
# "variables" & "default" are used by all jobs
variables:
TF_ROOT: '${CI_PROJECT_DIR}/terraform'
TF_CLI_CONFIG_FILE: .terraformrc
AWS_STS_REGIONAL_ENDPOINTS: regional
AWS_DEFAULT_REGION: eu-west-2
ASG_MODULE_PATH: module.aws_asg.aws_autoscaling_group.main_asg
default:
image:
name: hashicorp/terraform:light
entrypoint: ['']
cache:
paths:
- ${TF_ROOT}/.terraform
tags:
- nonlive # This tag matches the group wide GitLab runner.
before_script:
- cd ${TF_ROOT}
# List of all stages (jobs within the same stage are executed concurrently)
stages:
- init
- infraci_plan
- infraci_taint
- infraci_apply
- dev_plan
- dev_taint
- dev_apply
# "Hidden" jobs we use as templates to improve code reuse.
.default:
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
.plan:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_plan
script:
- terraform workspace select ${CI_ENVIRONMENT_NAME}
- terraform plan
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_ENVIRONMENT_NAME != "infraci"'
when: manual
allow_failure: false
.taint:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_taint
script: terrafrom taint ${ASG_MODULE_PATH}
needs:
- ${CI_ENVIRONMENT_NAME}_plan
.apply:
extends: .default
stage: ${CI_ENVIRONMENT_NAME}_apply
script: terraform apply -auto-approve
# Create actual jobs
## init - runs once per pipeline
init:
stage: init
script: terraform init
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
when: always
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "web"'
when: manual
## infraci - auto deploy
infraci_plan:
extends: .plan
environment:
name: infraci
infraci_taint:
extends: .taint
environment:
name: infraci
infraci_apply:
extends: .apply
environment:
name: infraci
## dev - manual deployment
dev_plan:
extends: .plan
environment:
name: dev
dev_taint:
extends: .taint
environment:
name: dev
dev_apply:
extends: .apply
environment:
name: dev
Unfortunately this fails validation with the following error:
infraci_plan job: chosen stage does not exist; available stages are .pre, init, infraci_plan, infraci_taint, infraci_apply, dev_plan, dev_taint, dev_apply, .post
My assumption is that it's to do with interpolating CI_ENVIRONMENT_NAME in the hidden jobs but not actually setting the value until the jobs where the jobs are actually defined.
If that's the case though what's a way to get the setup I need without a severe amount of duplication?
