11

I want to connect service from one GKE cluster to another one. I created service as a internal load balancer and I would like to attach a static ip to it. I created my service.yml

apiVersion: v1
kind: Service
metadata:
  name: ilb-service
  annotations:
    cloud.google.com/load-balancer-type: "Internal"
    kubernetes.io/ingress.global-static-ip-name: es-test
  labels:
    app: hello
spec:
  type: LoadBalancer
  selector:
    app: hello
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP

However after apply -f when I check the service the load balancer ingress looks like this:

status:
  loadBalancer:
    ingress:
    - ip: 10.156.0.60

And I cannot connect using the static ip. How to solve it ?

EDIT:

After suggestion I changed the yml file to:
apiVersion: v1
kind: Service
metadata:
  name: ilb-service
  annotations:
    cloud.google.com/load-balancer-type: "Internal"
  labels:
    app: hello
spec:
  type: LoadBalancer
  selector:
    app: hello
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
  loadBalancerIP: "xx.xxx.xxx.xxx" -- here my static ip

Service now looks like it:

spec:
  clusterIP: 11.11.1.111
  externalTrafficPolicy: Cluster
  loadBalancerIP: xx.xxx.xxx.xxx
  ports:
  - nodePort: 31894
    port: 80
    protocol: TCP
    targetPort: 8080
  selector:
    app: hello
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer: {}

And I still cannot connect

Clyde Barrow
  • 1,924
  • 8
  • 30
  • 60
  • does the GCP Loadbalancer that gets created have the same IP as the one reflected in your spec? – Patrick W Feb 05 '20 at 13:52
  • In fact, if your statius remains blank the way it is currently, it means the IP is not being assigned. Is the IP currently reserved as static? if so, that won't work. The IP needs to be available, if you have it reserved as static, the GCP platform does not see it as available. – Patrick W Feb 05 '20 at 13:53
  • Yes I reserved the ip as static I thought that was the whole clue. So what kind of ip I should put as a loadBalancerIP for it to work ? Just pick some random from my range ? Currently it describe my external ip as pending ``` NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ilb-service LoadBalancer 11.11.1.111 80:31894/TCP 28m ``` – Clyde Barrow Feb 05 '20 at 13:57
  • Yeah just use any unused up from the subnet. You can even release the currently reserved ip and use that. This is a common mistake and it's not clear in gcp doc's how this works. This is specifically for internal load balancers – Patrick W Feb 05 '20 at 14:00
  • thanks it works right now. But I wanted to be able to connect from one cluster to the load balancer by some kind of dns name not an ip. In the docs I've read that reserving a static ip and adding dns record is a solution. How to reach the load balancer by a dns name in your approach ? – Clyde Barrow Feb 05 '20 at 14:18
  • You can still configure DNS for the up in whatever you use as a DNS server. The up is static as long as it is in use – Patrick W Feb 05 '20 at 14:24

2 Answers2

13

November 2021 Update

It is possible to create a static internal IP and assign it to a LoadBalancer k8s service type.

  1. Go to the VPC networks -> Select your VPC -> Static Internal IP Addresses
  2. Click Reserve Static Address, then select a name for your IP and click Reserve. You can choose IP address manually here as well.
  3. In your Service YAML add the following annotation. Also make sure type is LoadBalancer and then assign the IP address.
...
annotations: 
  networking.gke.io/load-balancer-type: "Internal"
...
type: LoadBalancer
loadBalancerIP: <your_static_internal_IP>

This will spin up an internal LB and assign your static IP to it. You can also check in Static Internal IP Addresses screen that new IP is now in use by freshly created load balancer. You can assign a Cloud DNS record to it, if needed. Also, you can choose IP address "shared" during the reservation process so it can be used by up to 50 internal load balancers.

Assigning Static IP to Internal LB

Enabling Shared IP

Alex Ulyanov
  • 329
  • 3
  • 8
  • After doing reservation in VPC and then deploying svc I get `Error 409: IP_IN_USE_BY_ANOTHER_RESOURCE - IP '10.12.111.205' is already being used by another resource`. On the other hand, when I first create svc with static IP from given subnet and then try to add reservation in VPC - similard error. – wujt Apr 13 '22 at 09:31
  • I use SharedVPC^ – wujt Apr 13 '22 at 09:40
  • @wujt Might be an issue with an older k8s version. The above example worked on 1.20+. I also found an example of issue that is similar to yours. https://github.com/kubernetes/kubernetes/issues/66762#issuecomment-575573615 – Alex Ulyanov Apr 22 '22 at 20:04
  • We use 1.20.15 for now and the flow is the same as described in link you provided. But error persists. – wujt Apr 24 '22 at 13:53
  • For shared VPC, guide in my answer. – Abhishek May 23 '23 at 09:07
1

I had the same issue, and was continuously getting below error

Service event error

Warning  SyncLoadBalancerFailed  0s (x6 over 18s)   loadbalancer-controller
Error syncing load balancer: failed to get address by IP "x.x.x.100" after
reservation attempt, err: "googleapi: Error 404: Address with IP
\"x.x.x.100\" was not found in region \"europe-west1\"", reservation err:
"googleapi: Error 409: IP_IN_USE_BY_ANOTHER_RESOURCE - IP 'x.x.x.100' is
already being used by another resource."

Like me, if you're using Shared VPC, Create the Static IP in the service project and not on the host project.

That's the only difference from the previous answer. For more details refer Reserve a Static Internal IP while using Shared VPC

Abhishek
  • 763
  • 7
  • 18