0

How would I configure the service and/or ingress to handle bringing up and down hundreds of these deployments based on the following constraints:

  • This deployment is foo-1, subsequent ones would be named uniquely
  • Each deployment maps to 1 unique pod (the meta name and replicas handles this)
  • Each pod is accessible on 2 unique ports
  • Ports will be assigned, like when using NodePort
  • All pods should be accessible from the same IP
  • I can use a LoadBalancer, but not one for each deployment
  • If it makes sense to use something besides deployments, that is fine

  • Deployments will be created and deleted individually, not as a group

    apiVersion: apps/v1
kind: Deployment
metadata:
  name: foo-1
  labels:
    app: foo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: foo
  template:
    metadata:
      labels:
        app: foo
    spec:
      containers:
        - name: foo
          image: bar/baz:latest
          ports:
            - containerPort: 83
              name: listen
            - containerPort: 85
              name: serve

I'm using GKE which I believe has a firewall that's preventing me from accessing the nodes directly. I'm not opposed to disabling the firewall, but I'd prefer to do this through a LoadBalancer if possible.

Coder1
  • 101
  • 3
  • 2
    you need to check something like kustomize to generate your 100 of {deployments,services} and create your service with type ClusterIP, so local services and now to expose them, using a custom ingress controller like nginx-ingress, Now you can create many ingress rules to export your services. – c4f4t0r Apr 14 '20 at 09:04
  • I looked into this and the issue with a single ingress is that each path is limited to a single port. In theory, I'd have to run multiple subdomains for each proxy instance. Also, the problem with ClusterIP would also mean I need to manage port assignments myself. Thanks for the thoughts though. – Coder1 Apr 15 '20 at 16:13

1 Answers1

1

I would recommend you to start with moving your Deployment into some abstraction. You can make it with Helm Chart or Kustomize.

Having such abstraction will allow you to make a Service that matches the specific Deployment dynamically. Then, you can expose each Service in various of ways, including NodePort. (I would not recommend it. Think of ClusterIP, but it depends on your needs.)

For instance, Helm Chart allows you to range through ports.

limakzi
  • 191
  • 4
  • I'll have a little app issuing the creation of deployments and whatever services are needed. I like the NodePort service idea however it seems to be frowned upon and GKE seems to have this approach locked down anyway. ClusterIP would require me to assign ports ahead of time unless Helm is the solution for this. I'll look into this. – Coder1 Apr 15 '20 at 16:19