GCP internal load balancer using ingress resource on gke

Question

I'm trying to figure out what's the best way to set up an internal load balancer on GCP from a GKE cluster, especially how to be able to register an internal domain name using it.

When the load balancer is created, its frontend contains no "service label" that would allow to reach the lb using a deterministic domain name. If I create manually a second frontend containing an arbitrary value for this field, I'm able to resolve the generated domain name on the lb IP, everything works fine... I just don't get how I can add this field from a k8s resource.

I checked ingress gce repo and the "FrontendConfig" CRD doesn't expose this field...

What would be the best way to get that ? Use the nginx-ingress-controller ? Set up external DNS to manage my own DNS zones ? something else ?

Best,

Matthieu

PS : also one "weird" thing is that, even if I created a dedicated "proxy-only subnet" in the same region, as asked in the documentation, the lb IPs are in the cluster subnet

Edit : actually, the IPs can't be in the proxy-only subnet. If you follow the accepted answer below, be sure to pick an IP from another subnet.

score 1 · Accepted Answer · answered Aug 28 '21 at 07:15

I would prefer creating internal static IP address for the LB and creating internal DNS record and then finally confiurging the ingress setup with those.

These are the overall steps to be done for the Internal HTTP(S) LB to work:

Create a proxy-only subnet.
Reserve a static internal IP address.
Create a private DNS zone and DNS record for the reserved IP.
Expose your deployment using a NodePort service and annotate the service with NEG annotation.

# web-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: YOUR_DEPLOYMENT_SERVICE_NAME
  namespace: default
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
spec:
  ports:
    name: host1
    port: 80
    protocol: TCP
    targetPort: 9376
  selector:
    app: hostname
  type: NodePort

Create the ingress resource and use the ingress class gce-internal

# internal-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ilb-demo-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "gce-internal"
    kubernetes.io/ingress.regional-static-ip-name: "ADDRESS_NAME"
spec:
  rules:
  - host: INTERNAL_DNS_RECORD
    http:
      paths:
      - backend:
          service:
            name: YOUR_DEPLOYMENT_SERVICE_NAME
            port:
              number: 80

References:

thanks @ATEF, that's also what I figured out as the most reasonable thing to do, especially when I realized I could let GCP pick the actual IP address so I don't have to maintain anything on this side. thanks for the confirmation (and the doc links ;) ) — err0r500, Aug 28 '21 at 09:12

GCP internal load balancer using ingress resource on gke

1 Answers1