"default_secret_name" is no longer applicable for Kubernetes v1.24.0 and above

Question

I'm using Terraform workload-identity module , to create Kubernetes service account in Google Cloud. When i apply the changes, I'm getting below warning.

"default_secret_name" is no longer applicable for Kubernetes v1.24.0 and above │ │ with module.app-workload-identity.kubernetes_service_account_v1.main, │
on ../../modules/workload-identity/main.tf line 57, in resource "kubernetes_service_account_v1" "main": │ 57: resource "kubernetes_service_account_v1" "main" { │ │ Starting from version 1.24.0 Kubernetes does not automatically generate a token for service accounts, in this case, "default_secret_name" will be │ empty

Workload-Identity main.tf

locals {
  service_account_tmp = var.google_service_account_email== "" ? "projects/${var.project_id}/serviceAccounts/cloudsql-sa@${var.project_id}.iam.gserviceaccount.com" : var.google_service_account_email
  service_id = "projects/${var.project_id}/serviceAccounts/cloudsql-sa@${var.project_id}.iam.gserviceaccount.com"
  k8s_sa_gcp_derived_name = "serviceAccount:${var.project_id}.svc.id.goog[${var.namespace}/${local.output_k8s_name}]"
  gcp_sa_email            = var.google_service_account_email

  # This will cause terraform to block returning outputs until the service account is created
  k8s_given_name       = var.k8s_sa_name != null ? var.k8s_sa_name : var.name
  output_k8s_name      = var.use_existing_k8s_sa ? local.k8s_given_name : kubernetes_service_account.main[0].metadata[0].name
  output_k8s_namespace = var.use_existing_k8s_sa ? var.namespace : kubernetes_service_account.main[0].metadata[0].namespace
}

# resource "google_service_account" "cluster_service_account" {
  # GCP service account ids must be < 30 chars matching regex ^[a-z](?:[-a-z0-9]{4,28}[a-z0-9])$
  # KSA do not have this naming restriction.
  # account_id   = substr(var.name, 0, 30)
  # display_name = substr("GCP SA bound to K8S SA ${local.k8s_given_name}", 0, 100)
  # project      = var.project_id
# }
resource "kubernetes_namespace" "k8s_namespace" {
  metadata {
    name = var.namespace
  }
}

# resource "kubernetes_secret_v1" "main" {
#  metadata {
#    name      = var.name
#    namespace = var.namespace
#    annotations = {
#      "kubernetes.io/service-account.name"      = kubernetes_service_account_v1.main.metadata.0.name
#      "kubernetes.io/service-account.namespace" = kubernetes_service_account_v1.main.metadata.0.namespace
#    }
#    generate_name = "${kubernetes_service_account_v1.main.metadata.0.name}-token-"
#  }
#  type = "kubernetes.io/service-account-token"
#  wait_for_service_account_token = true
#}

resource "kubernetes_service_account" "main" {
  count = var.use_existing_k8s_sa ? 0 : 1
  metadata {
    name      = var.name
    namespace = var.namespace
    annotations = {
      "iam.gke.io/gcp-service-account" = var.google_service_account_email
    }
  }
}


module "annotate-sa" {
  source  = "terraform-google-modules/gcloud/google//modules/kubectl-wrapper"
  version = "~> 2.0.2"

  enabled          = var.use_existing_k8s_sa && var.annotate_k8s_sa
  skip_download    = true
  cluster_name     = var.cluster_name
  cluster_location = var.location
  project_id       = var.project_id

  kubectl_create_command  = "kubectl annotate --overwrite sa -n ${local.output_k8s_namespace} ${local.k8s_given_name} iam.gke.io/gcp-service-account=${local.gcp_sa_email}"
  kubectl_destroy_command = "kubectl annotate sa -n ${local.output_k8s_namespace} ${local.k8s_given_name} iam.gke.io/gcp-service-account-"
}

resource "google_service_account_iam_member" "main" {
  service_account_id = local.service_id
  role               = "roles/iam.workloadIdentityUser"
  member             = local.k8s_sa_gcp_derived_name
}

As per the this documentation , I have tried to add the resource "kubernetes_secret_v1" to create a service account token. But still getting the same warning message.

Answer 1

From this git issue kubernetes_service_account issue has been successfully fixed using this manifest.

I found this alternative solution where changes are made using the terraform resource kubernetes_manifest to manually generate the service accounts along with their secret.

Can you try the main.tf file and let me know if this works.

For more information follow this Issue.

Answer 2

We faced a similar issue very recently. What worked for us was defining the secret in the kubernetes_service_account_v1 terraform resource.

Here's an example:

resource "kubernetes_service_account_v1" "svc_test" {
   metadata {
      name = "svc-test"
      namespace = "test"
   }
   secret {
    name = "svc-test-token"
  }
}

resource "kubernetes_secret_v1" "svc_test_token" {
  metadata {
    name = "${kubernetes_service_account_v1.svc_test.metadata[0].name}-token"
    annotations = {
      "kubernetes.io/service-account.name" = kubernetes_service_account_v1.svc_test.metadata[0].name
    }
    namespace = "test"
    generate_name = "${kubernetes_service_account_v1.svc_test.metadata[0].name}-token"

  }
  type = "kubernetes.io/service-account-token"
  
  wait_for_service_account_token = true
}

We found that without this, the secret gets created with the serviceaccount reference but there is no reference of the secret in the serviceaccount.

Hope it helps.

Answer 3

According to the Terraform AWS provider documentation warning about the deprecation of default_secret_name in service_account_v1 resource, the kubernetes_secret_v1 resource should be used to created the secret to hold the Service Account token, since Service Account tokens are no longer created automatically in K8s versions starting 1.24 and beyond. These explicitly created tokens are only needed by legacy software, which are unable to use the new TokenRequest API (see K8s docs).

When you do need to explicitly create a Secret with the Service Account token, to support legacy behavior, the following works very well:

resource "kubernetes_service_account_v1" "this" {
  provider = kubernetes.minikube
  metadata {
    name      = local.service_account_name
    namespace = local.namespace
  }
}

resource "kubernetes_secret_v1" "this" {
  provider = kubernetes.minikube
  metadata {
    name      = "${kubernetes_service_account_v1.this.metadata[0].name}-token"
    namespace = local.namespace
    annotations = {
      "kubernetes.io/service-account.name" = kubernetes_service_account_v1.this.metadata[0].name
    }
  }
  type = "kubernetes.io/service-account-token"
}