Can not deploy helm chart in Terraform (Kubernetes cluster unreachable)

Question

I am started GKE cluster using Terraform (link), Now I am trying to release the helm charts on the cluster, and tried for "Nginx Ingress" helm chart which is as follow:

resource "helm_release" "ingress" {
  name       = "ingress"
  repository = "https://kubernetes.github.io/ingress-nginx"
  chart      = "ingress-nginx"
}

Terraform Plan:

Terraform will perform the following actions:

  # helm_release.ingress will be created
  + resource "helm_release" "ingress" {
      + chart            = "ingress-nginx"
      + disable_webhooks = false
      + force_update     = false
      + id               = (known after apply)
      + metadata         = (known after apply)
      + name             = "ingress"
      + namespace        = "default"
      + recreate_pods    = false
      + repository       = "https://kubernetes.github.io/ingress-nginx"
      + reuse            = false
      + reuse_values     = false
      + status           = "DEPLOYED"
      + timeout          = 300
      + verify           = false
      + version          = "2.3.0"
      + wait             = true
    }

But I am getting an error

Error: Kubernetes cluster unreachable: Get https://35.232.164.12/version?timeout=32s: dial tcp 35.232.164.12:443: i/o timeout

  on helm.tf line 36, in resource "helm_release" "ingress":
  36: resource "helm_release" "ingress" {

How is this error related to deploying multiple Helm charts? The error is saying you don't have network connectivity to the API server. — ydaetskcoR, May 27 '20 at 08:55
@ydaetskcoR, I am new to this terraform and I don't know much about it — Ravindra Gupta, May 27 '20 at 12:06

score 1 · Answer 1 · answered May 27 '20 at 12:38

Here terraform not able to create a connection with Kubernetes cluster. cluster is unreachable.

Also using terraform also have to first setup tiller then and then helm chart will work.

in terraform, you can use helm provider, if are using service account you add existing one or create one if required

provider "helm" {
  service_account = "${kubernetes_service_account.helm_account.metadata.0.name}"
  tiller_image = "gcr.io/kubernetes-helm/tiller:${var.helm_version}"
  #install_tiller = false

  kubernetes {
    host                   = "${google_container_cluster.<name>.endpoint}"
    token                  = "${data.google_client_config.current.access_token}"

    client_certificate     = "${base64decode(google_container_cluster.data-dome-cluster.master_auth.0.client_certificate)}"
    client_key             = "${base64decode(google_container_cluster.data-dome-cluster.master_auth.0.client_key)}"
    cluster_ca_certificate = "${base64decode(google_container_cluster.data-dome-cluster.master_auth.0.cluster_ca_certificate)}"
  }

one helm started running you can start setting up other releases using helm package manager.

you can also check this answer for more details : Deploying Helm workloads with Terraform on GKE cluster

This is for Helm2, I am looking for Helm3 – Ravindra Gupta May 28 '20 at 03:10 — Ravindra Gupta, May 28 '20 at 03:10
@RavindraGupta Added an example with Helm3 – jarv Sep 03 '21 at 12:26 — jarv, Sep 03 '21 at 12:26

score 1 · Answer 2 · answered Sep 03 '21 at 12:24

Here is an example with Helm3:

Note:

[cluster endpoint] and [ca certificate] are outputs of the cluster that was created with Terraform
You will need a cluster.admin and iam.serviceAccountTokenCreator roles on the service account

    data "google_service_account_access_token" "kubernetes_sa" {
      target_service_account = ""
      scopes                 = ["userinfo-email", "cloud-platform"]
      lifetime               = "3600s"
    }
    
    provider "kubernetes" {
      host  = "https://[cluster endpoint]"
      token = data.google_service_account_access_token.kubernetes_sa.access_token
      cluster_ca_certificate = base64decode(
        module.gitlab-gke.cluster_ca_certificate
      )
    }
    
    provider "helm" {
      kubernetes {
        host                   = "https://[cluster endpoint]"
        token                  = data.google_service_account_access_token.kubernetes_sa.access_token
    
        cluster_ca_certificate = base64decode(
          [ca certificate]
        )
      }
    }

Can not deploy helm chart in Terraform (Kubernetes cluster unreachable)

2 Answers2