2

We are working on an Infrastructure as Code for Hybrid Setup (GCP). As part of that we have to mount cloud network Filestore with on-premises Kubernetes cluster. We are maintaining two VPC networks in GCP for on-premises & cloud environments. We have configured the required Google cloud HA VPN that lets securely connect on-premises VPC network to cloud VPC. We have verified private connectivity over VPN from the instances in a VPC network, to reach instances in second network VPC. But we are unable to mount the Filestore through VPN.

Here is the kubectl describe pod command output.

Events:
  Type     Reason       Age                  From               Message
  ----     ------       ----                 ----               -------
  Normal   Scheduled    9m46s                default-scheduler  Successfully assigned axp/filestore-operations-755fd78fb6-w7z5d to gke-hybrid-onprem-kuber-app-node-pool-21f25437-nfvx
  Warning  FailedMount  57s (x4 over 7m43s)  kubelet            Unable to attach or mount volumes: unmounted volumes=[nfs-pvc], unattached volumes=[nfs-pvc kube-api-access-drvb5]: timed out waiting for the condition
  Warning  FailedMount  42s (x3 over 6m46s)  kubelet            MountVolume.SetUp failed for volume "xligw-pilot-pv" : mount failed: exit status 1
Mounting command: /home/kubernetes/containerized_mounter/mounter
Mounting arguments: mount -t nfs <ip_address>:/filestore_vol1 /var/lib/kubelet/pods/49bde489-bf00-47e1-94ba-eeccfb803ec1/volumes/kubernetes.io~nfs/xligw-pilot-pv
Output: Mount failed: mount failed: exit status 32
Mounting command: chroot
Mounting arguments: [/home/kubernetes/containerized_mounter/rootfs mount -t nfs <ip_address>:/filestore_vol1 /var/lib/kubelet/pods/49bde489-bf00-47e1-94ba-eeccfb803ec1/volumes/kubernetes.io~nfs/xligw-pilot-pv]
Output: mount.nfs: Connection timed out

Furthermore, I have advertised the IP address of the Filestore instance as a custom route during BGP configuration.

https://cloud.google.com/network-connectivity/docs/router/how-to/advertising-custom-ip

Appreciate if you can suggest a solution for this.

Kavindu Ariyasinghe
  • 41
  • 5
  • Did you check this documentation? https://cloud.google.com/filestore/docs/remote-mounting – guillaume blaquiere Nov 04 '21 at 08:53
  • You might first want to attempt to mount the Filestore volume directly on the host where you are running K8s on prem. Minimally, ssh into the worker nodes on prem and ensure you can reach the Filestore endpoint from the on-prem host. – Gari Singh Nov 05 '21 at 09:40
  • @guillaumeblaquiere Yes i did. I have established the VPN connection according to this architecture. [link](https://cloud.google.com/network-connectivity/docs/vpn/images/ha-vpn-gcp-to-on-prem-2-a.svg) I updated my question with kubectl describe pod command output. – Kavindu Ariyasinghe Nov 09 '21 at 09:40
  • @GariSingh I have advertised the IP address of the Filestore instance as a custom route during BGP configuration. I tried to reach the Filestore using ping command from the both on-prem and cloud worker nodes. i couldn't able to reach the Filestore from both environments. However i can mount the Filestore with the cloud Kubernetes cluster. – Kavindu Ariyasinghe Nov 09 '21 at 09:50
  • Are you trying to do [ on-prem -> VPN -> VPC -> Filestore ] or [ on-prem -> VPN -> VPC1 -> VPC2 -> Filestore ]? – Gari Singh Nov 09 '21 at 09:59
  • @GariSingh I'm maintaining two VPCs in the GCP for cloud and on-prem environments. Filestore is in the cloud environment VPC. I can mount the Filestore with k8 cluster in the cloud environment VPC. I need to mount it with the k8 cluster in the on-prem VPC. – Kavindu Ariyasinghe Nov 09 '21 at 10:26

0 Answers0