1

I would like to deploy an ngingx-ingress controller on my self-hosted Kubernetes (microk8s) that is configurable to listen on one or more interfaces (external IPs).

Not even sure if that is easily possible and I should just switch to an external solution, such as HAProxy or an nginx.

Required behavior:

192.168.0.1-H"domain.com":443/frontend -> 192.168.0.1 (eth0) -> ingress -> service-frontend
192.168.0.1-H"domain.com":443/backend -> 192.168.0.1 (eth0) -> ingress -> service-backend
88.88.88.88-H"domain.com":443/frontend -> 88.88.88.88 (eth1) -> ? -> [403 or timeout]
88.88.88.88-H"domain.com":443/backend -> 88.88.88.88 (eth1) -> ? -> [403 or timeout]

And then later the eth1 interface should be able to be switched on, so that requests on that interface behave the same as on eth0.

I would like to be able to deploy multiple instances of services for load-balancing. I would like to keep the configuration in my namespace (if possible) so I can always delete and apply everything at once.

I'm using this guide as a reference: https://kubernetes.github.io/ingress-nginx/deploy/baremetal/

I was able to get something working with minikube, but obviously could not expose any external IPs and performance was quite bad. For that, I just configured a "kind: Ingress" and that was it.

So far, the ingress controller that's default on microk8s seems to listen on all interfaces and I can only configure it in its own namespace. Defining my own ingress seems to not have any effect.

Harsh Manvar
  • 27,020
  • 6
  • 48
  • 102
brainwash
  • 690
  • 5
  • 19

3 Answers3

1

I would like to deploy an ngingx-ingress controller on my self-hosted Kubernetes (microk8s) that is configurable to listen on one or more interfaces (external IPs).

For above scenario, you have to deploy the multiple ingress controller of Nginx ingress and keep the different class name in it.

Official document : https://kubernetes.github.io/ingress-nginx/user-guide/multiple-ingress/

So in this scenario, you have to create the Kubernetes service with Loadbalancer IP and each will point to the respective deployment and class will be used in the ingress object.

If you looking forward to use the multiple domains with a single ingress controller you can easily do it by mentioning the host into ingress.

Example for two domain :

  1. bar.foo.dev
  2. foo.bar.dev

YAML example

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: frontdoor-bar
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  tls:
    - hosts:
        - bar.foo.dev
      secretName: tls-secret-bar
  rules:
    - host: bar.foo.dev
      http:
        paths:
          - backend:
              serviceName: barfoo
              servicePort: 80
            path: /(.*)
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: frontdoor-foo
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  tls:
    - hosts:
        - foo.bar.dev
      secretName: tls-secret-foo
  rules:
    - host: foo.bar.dev
      http:
        paths:
          - backend:
              serviceName: foobar
              servicePort: 9000
            path: /(.*)
Harsh Manvar
  • 27,020
  • 6
  • 48
  • 102
  • That's very useful, but I don't see how the IP address selection happens. Was thinking of something like MetalLB and the annotation "metallb.universe.tf/address-pool". So in your example barfoo is on IP1 and foobar on IP2? – brainwash Sep 13 '21 at 04:45
  • No, you are right you have to add annotation as you are on baremetal and using metalLB, for above example when controller get deployed along with the service get deployed with type `Loadbalancer` so by default cloud provider attach IP to it and expose it. – Harsh Manvar Sep 13 '21 at 04:48
0

One potential fix was much simpler than anticipated, no messing with MetalLB needed or anything else.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    kubernetes.io/ingress.class: "public"
    nginx.ingress.kubernetes.io/whitelist-source-range: 192.168.0.0/24
...

This does not answer the question of splitting an Ingress across multiple interfaces, but it does solve the problem of restricting public access. By default, bare-metal ingress will listen on all interfaces, which might be a security issue.

brainwash
  • 690
  • 5
  • 19
0

This solution works without enabling ingress on Microk8s:

kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/service-upstream: 'true'
    nginx.ingress.kubernetes.io/rewrite-target: "/$2"
  name: ingress-resource
  namespace: namespace-name
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: service-name
            port:
              number: service-port
        path: /namespace-name/service-name(/|$)(.*)
        pathType: Prefix

  • kubectl get svc -n ingress-nginx

  • now get either CLUSTER-IP or EXTERNAL-IP and :

    curl ip/namespace-here/service-here

hakik ayoub
  • 55
  • 1
  • 7
  • Thank you for the solution, but how does it differentiate between request coming in from different interfaces/IPs? – brainwash Sep 15 '21 at 10:48
  • that's what we call Ingress. One Ip (ingress controller) multiple ingress resources (rules to indicate to which service the request must go). When you create another deployment with its service, you must create an ingress resource to let the ingress controller knows about it. I hope this gives an answer – hakik ayoub Sep 15 '21 at 11:11
  • Thank you, but how do you create different ingresses for different IP addresses? – brainwash Sep 16 '21 at 09:25
  • It seems you didn't read about ingress. In my example we have one ingress controller and multiple ingress resources(rules to direct the traffic to a specific service). so when you curl on your "ingress-resource-ip/default-namespace/service-1" it will redirect you to service-1. If you curl to "ingress-resource-ip/default-namespace/service-2" it will direct you to service-2. – hakik ayoub Sep 16 '21 at 10:23