Sharing resources across terraform environments

Question

I have created a simple terraform infrastructure that will created some resource on aws.I have each environment variables in a separate tfvars. However I do have shared resources like security groups that would be used across environments.

How can I structure the code in such a way that I can create/destroy environments without destroying the shared resources?. I could create separate workspaces but since all resources are defined in same repo, if i destroy one environment, I am going to destroy the shared resources in the process.

A simplified version of my directory structure is as follows:

/
/main.tf
/vars/dev.tfvars
/vars/qa.tfvars
/vars/prod.tfvars

Moreover when using modules how can i specify to only delete part of it during a terraform destroy

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = "my-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_nat_gateway = true
  enable_vpn_gateway = true

  tags = {
    Terraform = "true"
    Environment = "dev"
  }
}

For example if i delete dev environment , i just want the subnets to be deleted and not the vpc since i have one vpc shared between all environments.

you have changed your question based on my answer to use shared resources, so my answer is outdated — Hatim, Dec 22 '21 at 15:50
Ideally you would split the environments by workspaces, and separate out the shared resources into their own module. However, since the shared resources and environment resources are so tightly coupled in your environment and in your config, then you may be in a tough spot here. Also note that not using `terraform destroy` and instead using `terraform plan/apply` with config updates as normal would solve your problem around selective resource removal. — Matthew Schuchard, Dec 22 '21 at 15:50
In other words: you either need to re-architect your infrastructure to de-couple the shared and environment resources, or re-architect the config to de-couple shared resources with environment resources within the same module. — Matthew Schuchard, Dec 22 '21 at 15:53

Hatim · Accepted Answer · 2021-12-22T15:57:18.993

For your need, you can create another TF State which contains your shared resources only, Then import it as a variable or Data in your current State. You can do as follow

/Shared-resources   <== shared stack to export SG
   /main.tf
   /export.tf
/Stack   <== Import SG here 
  /main.tf
  /vars/dev.tfvars
  /vars/qa.tfvars
  /vars/prod.tfvars

And your create 2 differents pipelines CICD to deploy "shared-resource" in his TF state, and the others in differents TF states.

Edit based on question changes:

First, it's not a good practice to have a unique VPC across many env, in your case you can have one VPC per environment, that solves the problem from the beginning :).

What you can do, is to extract the resources created in another state and leave the VPC untouched, that's mean extract the subnets and all network stuff in the shared-service state. Then import it in your VPC State as Data or variables from the other shared resources State

yes you have changed your question based on my answer to use shared resources, so my answer is outdated – — Hatim, Dec 22 '21 at 15:52

score 0 · Answer 2 · answered Jan 19 '22 at 02:26

0

Another solution could be to use terraform workspaces to define each environment. See answer here: Sharing resources between Terraform workspaces

answered Jan 19 '22 at 02:26

intotecho

4,925
3
39
54

Sharing resources across terraform environments

2 Answers2