Create and update secrets on AWS if they already exists with Terraform

Question

I am trying to create secrets for a microservice using Terraform, and I want to initialize the secrets with some values from my Terraform configuration, but I also want to allow adding secrets manually.

This is what I initially tried.

locals {
  secret_name = "${var.environment}/microservices/${var.project_name}/secrets"
}

resource "aws_secretsmanager_secret" "secrets" {
  name        = local.secret_name
  description = "Secrets for ${var.project_name} microservice."
}

resource "aws_secretsmanager_secret_version" "credentials" {
  secret_id = aws_secretsmanager_secret.secrets.id

  secret_string = jsonencode(
    {
      ENV            = var.environment,
      DYNAMODB_TABLE = var.dynamo_db_table_name
      AWS_ACCESS_KEY = var.aws_key
      AWS_SECRET_KEY = var.aws_secret
    }
  )
}

the problem with the code above is that if a manual secret is added, the code with override those secrets, therefore I updated my code to fetch the secrets and merge them together, as shown below.

data "aws_secretsmanager_secret_version" "secret-version" {
  secret_id = aws_secretsmanager_secret.secrets.id
}

resource "aws_secretsmanager_secret_version" "credentials" {
  secret_id = aws_secretsmanager_secret.secrets.id

  secret_string = jsonencode(merge(
    try(jsondecode(data.aws_secretsmanager_secret_version.secret-version.secret_string), {}),
    {
      ENV            = var.environment,
      DYNAMODB_TABLE = var.dynamo_db_table_name
      AWS_ACCESS_KEY = var.aws_key
      AWS_SECRET_KEY = var.aws_secret
    }
  ))
}

the above code was working fine, but then I was working with a second environment, and I encounter an issue. I cannot fetch data from a secret that does not exist.

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Secrets Manager Secret "arn:aws:secretsmanager:us-east-2:***:secret:staging/microservices/***/secrets-HRCPqx" Version "AWSCURRENT" not found
│ 
│   with module.ms_secrets.data.aws_secretsmanager_secret_version.secret-version,
│   on ../modules/secrets/main.tf line 15, in data "aws_secretsmanager_secret_version" "secret-version":
│   15: data "aws_secretsmanager_secret_version" "secret-version" {
│ 
╵

So basically my question is, how can I allow the creation of a secret with default values or empty and then fetch data from an already created secret and merge them together?

Terraform does not do *"... if resource does not exist yet"*. A data source expects a thing to be there and a resource creates one and assume it was not already there. The only way around this is to [import existing resources](https://developer.hashicorp.com/terraform/language/import) into your state and drop the *"... if not exist"* logic. — luk2302, Jun 13 '23 at 16:20

score 2 · Accepted Answer · answered Jun 13 '23 at 16:59

The best practice is to include the ignore_changes on the secret_string field. This configuration ensures that secrets are initially created using the default value. The manual changes made to the environment will not be overridden during subsequent code releases.

resource "aws_secretsmanager_secret_version" "credentials" {
  secret_id = aws_secretsmanager_secret.secrets.id

  secret_string = jsonencode(
    {
      ENV            = var.environment,
      DYNAMODB_TABLE = var.dynamo_db_table_name
      AWS_ACCESS_KEY = var.aws_key
      AWS_SECRET_KEY = var.aws_secret
    }
  )

  lifecycle {
    ignore_changes = [secret_string]
  }
}

p.s. This way, you will have to update the secret values manually to all the environments.

I believe that this could work as well. Thank you! – Abraham Jun 15 '23 at 09:33 — Abraham, Jun 15 '23 at 09:33

Create and update secrets on AWS if they already exists with Terraform

1 Answers1