The problem I am trying to solve:
I want to achieve the following setup:
- Terraform provisioned infrastructure consisting of Kubernetes cluster, DNS records pointing to that cluster,
- Kubernetes services, deployments configured through Kubernetes
.yaml
files, - Kubernetes ingress service acting as a gateway to the services on the cluster.
How I am trying to do this:
- Add kubernetes cluster and kubernetes ingress through Terraform. Bind kubernetes ingress IP to DNS records.
- Deploy all the services with
kubectl
.
The relevant part of terraform code for this looks something like this:
resource "kubernetes_ingress" "main-ingress" {
metadata {
name = "main-ingress"
}
spec {
rule {
host = "service.example.com"
http {
path {
backend {
service_name = "service-name"
service_port = 8080
}
path = "/"
}
}
}
}
wait_for_load_balancer = true
}
resource "digitalocean_record" "subdomain_link" {
domain = "example.com"
type = "A"
name = "service"
value = kubernetes_ingress.main-ingress.load_balancer_ingress[0].ip
}
The problem is that kubernetes_ingress
does not seem to be able to be deployed without managing the pods, services and deployments through Terraform. I want to manage them separately through kubectl
kustomize files.
The reason I am trying to create kubernetes ingress through Terraform is that I need the assigned IP for the DNS records in terraform.
The question I have is as follows:
Can I deploy kubernetes ingress through terraform without managing the kubernetes service through terraform?
If I can't, is there any way to know what IP will be assigned to the ingress during terraform apply so that I can bind the DNS to it?
P.S. Sorry for the convoluted question, not sure how I can structure it better.