0

We are building a new Kubernetes cluster in GCP, using a Shared VPC. The cluster was created with VPC-native (i.e., alias IP) enabled. Everything about the Shared VPC seems to be in order. The node, pod, and service IP addresses are all as expected from the Shared VPC. I even have one service on an internal load balancer on the Shared VPC subnet which works just fine.

But when I run the simplest of tests for an external service

kubectl create deployment hello-web --image=nginxdemos/hello
kubectl expose deployment hello-web --type=LoadBalancer --port 80 --target-port 80
kubectl get service hello-web

the public IP indicated by the last command, gets "This site can't be reached, x.x.x.x took too long to respond" from Chrome.

I ran the above test in our existing clusters which are not VPC-native nor Shared VPC and it works. So what might I be missing?

jlar310
  • 131
  • 2
  • Could you provide steps which you followed? Did you create a firewall rule to allow this traffic? When you created Cluster/VM in second project, did you set in Firewall > Netowrkking > Networking interfaces? – PjoterS Jun 08 '20 at 16:08

1 Answers1

2

When a GKE cluster uses a network from within the same project, the permissions are automatically there to allow the Kubernetes cluster service account to create firewall rules within that project. So it just works.

With a Shared VPC, the service project (GKE) needs to be given permission to create firewall rules in the host project. Or you must create the firewall rules manually. If the GKE cluster fails to create the firewall rules, it will be logged in the Kubernetes event long, along with the commands required to create the rules.

My solution was to grant role compute.securityAdmin in the Shared VPC host project to the service account in the GKE service project: service-[project-number]@container-engine-robot.iam.gserviceaccount.com.

The role above may be an overreach. At the very least, the service account needs the compute.firewalls.* rules within that role.

jlar310
  • 131
  • 2
  • For addition, you can also check GKE docs about [SharedVPC](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-shared-vpc) – PjoterS Jun 10 '20 at 10:36