Install two traefik ingress controller on same kubernetes Cluster

Question

I have a situation, where I am planning to use two separate traefik ingress controller inside the Kubernetes cluster.

I have a few URLs which I want to be accessible through VPN only, and few which can be publicly accessible.

In the current architecture, I have one traefik-ingress controller, and two separate ALBs, one internal and one internet-facing, both pointing to traefik.
Let's say, I have a URL public.example.com and private.example.com. public.example.com is pointing to internet-facing ALB, and private.example.com is pointing to internal ALB. But what if someone get to know the pointing of public.example.com and points private.example.com to same pointing in his /etc/hosts, he will be able to access my private website.

To avoid this, I am planning to run two separate traefik-ingress-controller, one which will be serving only private URL and one public URL. Can this be done? Or is there any other way to avoid this

score 7 · Accepted Answer · answered Apr 28 '20 at 19:48

To deploy two separate traefik-ingress controller, to serve private and public traffic separately, I used kubernetes.ingressclass=traefik args.

This is what documentation has to say for kubernetes.ingressclass:

--kubernetes.ingressclass  Value of kubernetes.io/ingress.class annotation to watch for

I created two deployment, having separate value for kubernetes.ingressclass.

One with kubernetes.ingressclass=traefik, which was behind a public ALB and kubernetes.ingressclass=traefik-internal, which was behind a private/internal ALB

For services, which I want to serve privately, I use the following annotations in ingress objects :

annotations:
    kubernetes.io/ingress.class: traefik-internal

and for public

annotations:
  kubernetes.io/ingress.class: traefik

My deployment.yaml

---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: traefik-internal-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-internal-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-internal-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-internal-ingress-lb
    spec:
      serviceAccountName: traefik-internal-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik:v1.7
        name: traefik-internal-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
        - --accesslog=true
        - --kubernetes.ingressclass=traefik-internal ##this makes it to watch only for ingress objects with annotaion "kubernetes.io/ingress.class: traefik-internal"

Hope this helps someone.

hi, what happens if an ingress it doesn't declare the annotation? — Cocuba, Jul 08 '22 at 21:16
ok what happens when you define a middleware? is it applied on public or private treafik? or we need to use ingressClass there as well? — Ijaz Ahmad, Sep 02 '22 at 05:50

score 4 · Answer 2 · answered Mar 29 '20 at 12:36

You can achieve this with a single Ingress controller inside the cluster but by creating various Ingress Kubernetes Objects.

For Private site:- consider whitelist-source-range annotation in the ingress resource.

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#whitelist-source-range

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: 10.0.0.0/24,172.10.0.1
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          serviceName: test
          servicePort: 80

For Public site:-

https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          serviceName: test
          servicePort: 80

This answer is somewhat correct for some situations, but it doesn't solve my problem. Because in case of ALB, may it be internal or internet-facing, the requests all come from private-IP only(of ALB). And whitelisting private IPs does not work. Though thanks for your answer, got to learn something new. — kadamb, Apr 28 '20 at 19:31

score 0 · Answer 3 · answered Mar 29 '20 at 13:01

Multiple Træfik Deployments can run concurrently in the same cluster.For instance, it is conceivable to have one Deployment deal with internal and another one with external traffic.

For such cases, it is advisable to classify Ingress objects through a label and configure the labelSelector option per each Træfik Deployment accordingly. To stick with the internal/external example above, all Ingress objects meant for internal traffic could receive a traffic-type: internal label while objects designated for external traffic receive a traffic-type: external label. The label selectors on the Træfik Deployments would then be traffic-type=internal and traffic-type=external, respectively.

Install two traefik ingress controller on same kubernetes Cluster

3 Answers3