Install Istio using Istio Operator and Terraform on EKS

Question

I'm new to Terraform. I need to set up Istio on the AWS EKS cluster. I thought of using Istio-Operator along with Terraform to do the same.

Below is the shell script to install Istio on EKS using Istio-Operator:

install-istio.sh

# Download and install the Istio istioctl client binary

# Specify the Istio version that will be leveraged throughout these instructions
ISTIO_VERSION=1.7.3

curl -sL "https://github.com/istio/istio/releases/download/$ISTIO_VERSION/istioctl-$ISTIO_VERSION-linux-amd64.tar.gz" | tar xz

sudo mv ./istioctl /usr/local/bin/istioctl
sudo chmod +x /usr/local/bin/istioctl

# Install the Istio Operator on EKS
istioctl operator init

# The Istio Operator is installed into the istio-operator namespace. Query the namespace.
kubectl get all -n istio-operator

# Install Istio components
istioctl profile dump default

# Create the istio-system namespace and deploy the Istio Operator Spec to that namespace.
kubectl create ns istio-system
kubectl apply -f istio-operator.yaml

# Validate the Istio installation
kubectl get all -n istio-system

Below is the istio-operator.yaml file used by install-istio.sh

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  namespace: istio-system
  name: istio-control-plane
spec:
  # Use the default profile as the base
  # More details at: https://istio.io/docs/setup/additional-setup/config-profiles/
  profile: default
  # Enable the addons that we will want to use
  addonComponents:
    grafana:
      enabled: true
    prometheus:
      enabled: true
    tracing:
      enabled: true
    kiali:
      enabled: true
  values:
    global:
      # Ensure that the Istio pods are only scheduled to run on Linux nodes
      defaultNodeSelector:
        beta.kubernetes.io/os: linux
    kiali:
      dashboard:
        auth:
          strategy: anonymous

Below is the main.tf file which executes the script

resource "null_resource" "install_istio" {

 provisioner "local-exec" {

    command = "/bin/bash install-istio.sh"
  }
}

I request you to help me with few queries:

How can I make use of the above script along with Terraform to install Istio on EKS cluster. What is the terraform part I need to include along with the above script?
Is there any missing part in the script. Will I face any problem updating the Istio using the above script?
What are the other parameters I need to include so that the script can install Istio on the EKS cluster?
How can I create Terraform module using the above script?

Thank you very much for your time. Appreciate all your help!

score 3 · Accepted Answer · answered Apr 15 '21 at 13:27

3

I believe you will encounter problems if using a local-exec provisioner like this.

Terraform does not play nice with resources it cannot reconcile. Especially when it comes to CRDs. Also, every time you will run terraform apply, you will run istioctl init over and over, which is probably not what you want.

What you can do, is to

convert the istio-operator to standard kubernetes manifests using

mkdir -p istio-operator
istio-operator dump > istio-operator/manifests.yaml

Create a istio-operator/kustomization.yaml file with

#istio-operator/kustomization.yaml

resources:
- manifests.yaml

Install the terraform kustomization provider

# terraform.tf

terraform {
  required_providers {
    kustomization = {
      source  = "kbst/kustomization"
      version = "0.4.3"
    }
  }
}

provider "kustomization" {
  // See online documentation on how to configure this
}

Install istio-operator with the terraform kustomization provider

# istio-operator.tf

data "kustomization" "istio_operator" {
  path     = "./istio-operator"
}

resource "kustomization_resource" "istio_operator" {
  for_each = data.kustomization.istio_operator.ids
  manifest = data.kustomization.istio_operator.manifests[each.value]
}

Create a IstioOperator manifest in istio/manifest.yaml

# istio/manifest.yaml

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: istio-control-plane
...

Create a istio/kustomization.yaml with

# istio/kustomization.yaml

resources:
- manifest.yaml

Install the IstioOperator with a second kustomization resource using terraform.

# istio.tf

data "kustomization" "istio" {
  path     = "./istio"
}

resource "kustomization_resource" "istio" {
  for_each = data.kustomization.istio.ids
  manifest = data.kustomization.istio.manifests[each.value]
  depends_on = [kustomization_resource.istio_operator]
}

I would recommend putting this whole thing in a separate folder, such as this

/home
  /project
    /terraform
      /istio
        terraform.tf
        istio_operator.tf
        istio.tf
        /istio
          kustomization.yaml
          manifest.yaml
        /istio-operator
          kustomization.yaml
          manifest.yaml

answered Apr 15 '21 at 13:27

Ludovic C

2,855
20
40

Thank you for your answer. Every time we run terraform apply, it will run istioctl init over and over. However, What do you think if we always run terraform destroy before applying terraform apply? – Sweta Sharma Apr 15 '21 at 13:46
1

No that would not solve it, since terraform doesn't know how to delete a `null_resource`. Running `terraform-apply` over and over *would* work, but you will lose precious time. You could also just install istio manually with `istioctl`. – Ludovic C Apr 15 '21 at 13:52
Thanks, Ludovic for your answer. However, I need to automate the Istio installation on the EKS cluster thought of creating Terraform module. What's your opinion? – Sweta Sharma Apr 15 '21 at 13:56
If you need to automate, then I would recommend the approach specified in my answer. Print out the manifests for istio-operator and istio using istioctl, and use the terraform `kustomize` provider to install those manifests. I'm following this approach for the clusters I manage. Personally, I try to avoid installing as much kubernetes stuff with terraform. I only create `kubernetes_namespaces`. I use terraform to install `argocd` with the `kustomize provider, and use `argocd` to install *all* of the kubernetes resources I need. It handles kubernetes much better than terraform. – Ludovic C Apr 15 '21 at 14:00
1

The `kubernetes-alpha` terraform provider will eventually be not experimental, perhaps it will makes things easier in the future to install kubernetes resources using terraform. – Ludovic C Apr 15 '21 at 14:03
1

Thanks, Ludovic for answering and sharing knowledge. – Sweta Sharma Apr 15 '21 at 14:04
1

You're welcome. If you encounter "chicken or egg" errors (cannot install IstioOperator before the istio CRDs are installed), then just use 2 different terraform modules that you would `terraform apply` separately. One for istio-operator, and the other one for the `Istio control plane`. – Ludovic C Apr 15 '21 at 14:07
I'm curious to know your opinion on installing Istio using Helm and Terraform because terraform has a provider for Helm. – Sweta Sharma Apr 16 '21 at 07:23
1

You could also do that. I dislike to use Helm because I prefer to be able to configure the manifests manually. It's difficult with helm, and much easier if you print out the full manifests (using either `istioctl dump` or `helm template`). That's what I do for all my enterprise projects. – Ludovic C Apr 16 '21 at 08:34
Thanks so much Ludovic for sharing knowledge! – Sweta Sharma Apr 17 '21 at 04:03
Hi Ludovic, When I try to run main.tf file it gives an error ``` module.istio_module.null_resource.install_istio (local-exec): Executing: ["/bin/sh" "-c" "/bin/bash install-istio.sh"] module.istio_module.null_resource.install_istio (local-exec): /bin/bash: install-istio.sh: No such file or directory``` Could you please help me missing part. Thanks – Sweta Sharma Apr 27 '21 at 11:08

Install Istio using Istio Operator and Terraform on EKS

1 Answers1