2

First for my question, I need to talk a bit about my enviroments:

  1. Google Basic Setup: 1x f1-micro instance with 3 nodes

  2. Kubernetes Setup: nginx-ingress-controller, cert-manager, 1-backend service with deployment, 1-frontend service with deployment.

  3. Mongo Atlas Setup: 3-replicaSet

Setup should not be a prolbem, but It might give some scenario feelings.

OK, Let comes to the issue, my Nodejs backend use the following url to connect to MonglAtlas database:

MONGODB_URI=mongodb+srv://username:pwd@mongotoken-66gqa.gcp.mongodb.net/test?retryWrites=true

IP Whitelist is my static public IP that use nginx-ingress to route. Let me define my.domain to my frontend webpage, and my.domain/api/ to backend api.

Everything is fine when IP Whitelist is ALLOW ACCESS FROM ANYWHERE, and backend could connect to MongoAtlas DB for no doubt.

But when I delete that option, and add the IP that matched with my.domain (double check, I ping my.domain is absolutely same IP), and then backend could not find the database with following error:

MongoNetworkError: connection 4 to closed https....

If there is something missing infos, please let me know. Any advice is appreciated!

Another suspected is that I got 1 static IP and 3 ephemeral IP in VPC network. I guess It means 3 node with loadbalancer IP. If the backend use ephemeral IP to connect to MongoAtlas backend, I must check the pod that in which nodes and make that node static, but this make no sense for Kubernetes. I hope there is another solution :(

Tokenyet
  • 4,063
  • 2
  • 27
  • 43
  • This is the similar issue on [serverfault](https://serverfault.com/questions/835425/kubernetes-external-connection-through-single-ip?newreg=8bb979ada50c4774a42b7f88424709b1), but have no more response. – Tokenyet Feb 02 '19 at 12:18
  • [Official Discussion](https://groups.google.com/forum/#!topic/kubernetes-users/C34yKt0qKtY), [Related Issue](https://stackoverflow.com/questions/41133755/static-outgoing-ip-in-kubernetes) – Tokenyet Feb 02 '19 at 12:53
  • [Official Documentation](https://cloud.google.com/solutions/using-a-nat-gateway-with-kubernetes-engine) NAT way. – Tokenyet Feb 02 '19 at 13:03
  • [KubeIP](https://github.com/doitintl/kubeip) – Tokenyet Feb 02 '19 at 13:09

2 Answers2

1

The solution I used is NAT. The concept is to establish 1- Google Compute Engine Instance as NAT Gateway, and mapping all the egress to a static ip. Oh, the most important is all the steps above do not need to manual config, just follow the documentation, and everything should be work as expected.

If there is a STATIC_ADDRESS QUOTA problem, you could change your ZONE and REGION to any QUOTA-remained area. For my case, us-central as NAT and us-west as Original Service.

Tokenyet
  • 4,063
  • 2
  • 27
  • 43
0

Ingress and egress traffic goes different ways.

When you connect to ingress, your traffic goes thru load balancer with static IP (I hope you use ingress with service type "Load Balancer).

When you connect from cluster to any external resource, you connect directly from the node when your container works, so you need to whitelist addresses of your nodes on Atlas to allow connections from them.

Anton Kostenko
  • 8,200
  • 2
  • 30
  • 37
  • As I mentioned, make all nodes to static IP (GCP) or access from anywhere (MongoAtlas). I just don't satisfied with both ways, but If this is the only solution, I would choose the MongoAtlas way. – Tokenyet Feb 02 '19 at 12:43