Suppose I create multiple ingress objects in k8s which point to the same service, same path and are exactly the same, only they have different names eg-. ingress-1 and ingress-2. How are the requests handled in this case? Are the requests duplicated or either ingress handles the request?
PS;- I know this doesn't make much sense but I am testing something out.
Let's check how it works!
I have default nginx deployment
# kubectl -n test get pods -o wide | grep nginx
nginx-65f88748fd-6w6fj 1/1 Running 0 26h 10.8.253.25 k8s-vm02 <none> <none>
nginx-65f88748fd-8fp7p 1/1 Running 0 26h 10.8.252.205 k8s-vm01 <none> <none>
nginx-65f88748fd-c7j29 1/1 Running 0 26h 10.8.253.24 k8s-vm02 <none> <none>
nginx-65f88748fd-frsbq 1/1 Running 0 26h 10.8.252.201 k8s-vm01 <none> <none>
nginx-65f88748fd-p4zvm 1/1 Running 0 26h 10.8.252.204 k8s-vm01 <none> <none>
nginx-65f88748fd-pd8gv 1/1 Running 0 25h 10.8.253.27 k8s-vm02 <none> <none>
nginx-65f88748fd-rkcjl 1/1 Running 0 26h 10.8.252.206 k8s-vm01 <none> <none>
nginx-65f88748fd-rn49k 1/1 Running 0 26h 10.8.253.26 k8s-vm02 <none> <none>
nginx-65f88748fd-w9dz8 1/1 Running 0 26h 10.8.252.203 k8s-vm01 <none> <none>
nginx-65f88748fd-xh42v 1/1 Running 0 25h 10.8.253.28 k8s-vm02 <none> <none>
service
# kubectl -n test get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 10.8.254.218 <none> 80/TCP 12d
and 2 similar Ingress resources in the same namespace
# kubectl -n test get ing
NAME HOSTS ADDRESS PORTS AGE
test-ingress-1 nginx.test.com 80 20m
test-ingress-2 nginx.test.com 80 20m
Their YAML's:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
namespace: test
name: test-ingress-1
annotations:
kubernetes.io/ingress.class: "service"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: nginx.test.com
http:
paths:
- path: /foo
backend:
serviceName: nginx
servicePort: 80
and
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
namespace: test
name: test-ingress-2
annotations:
kubernetes.io/ingress.class: "service"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: nginx.test.com
http:
paths:
- path: /foo
backend:
serviceName: nginx
servicePort: 80
one more thing about ingress.class: "service" - there are more then 1 ingress controllers in my environment and this particular ingress-controller
nginx-ingress-service-6gkhh 1/1 Running 0 4m20s 10.8.255.243 k8s-vm02 <none> <none>
has been created specially for demonstrating this example, so pay no attention for it
Anyway, is Ingress resource nginx.test.com/foo working now?
# curl -H "Host: nginx.test.com" http://10.8.255.243:80/foo
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Yes, it is. What's under the hood? Nginx config inside ingress-controller has only one server_name nginx.test.com no matter how many similar Ingress resources we have
# kubectl -n kube-system exec nginx-ingress-service-6gkhh -- cat /etc/nginx/nginx.conf
...
## start server nginx.test.com
server {
server_name nginx.test.com ;
listen 80;
listen [::]:80;
set $proxy_upstream_name "-";
location ~* ^/foo\/?(?<baseuri>.*) {
...
proxy_pass http://test-nginx-80;
...
## end server nginx.test.com
upstreams:
upstream test-nginx-80 {
keepalive 32;
server 10.8.252.203:80 max_fails=0 fail_timeout=0;
server 10.8.253.26:80 max_fails=0 fail_timeout=0;
server 10.8.252.201:80 max_fails=0 fail_timeout=0;
server 10.8.253.25:80 max_fails=0 fail_timeout=0;
server 10.8.252.204:80 max_fails=0 fail_timeout=0;
server 10.8.253.24:80 max_fails=0 fail_timeout=0;
server 10.8.252.205:80 max_fails=0 fail_timeout=0;
server 10.8.252.206:80 max_fails=0 fail_timeout=0;
server 10.8.253.27:80 max_fails=0 fail_timeout=0;
server 10.8.253.28:80 max_fails=0 fail_timeout=0;
}
Let's delete the test-ingress-1 Ingress resource
# kubectl -n test delete ing test-ingress-1
ingress.extensions "test-ingress-1" deleted
Ingress is still working:
# curl -H "Host: nginx.test.com" http://10.8.255.243:80/foo
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
}
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
So, you can create as many similar Ingress resources as you want (In case of using nginx-ingress-controller)
UPD: Let's create one more deployment in the same namespace:
apiVersion: apps/v1
kind: Deployment
metadata:
name: apache
namespace: test
spec:
replicas: 3
selector:
matchLabels:
app: apache
template:
metadata:
labels:
app: apache
spec:
containers:
- name: frontend
image: httpd
ports:
- containerPort: 80
and one more service
apiVersion: v1
kind: Service
metadata:
name: apache
namespace: test
spec:
ports:
- name: 80-80
port: 80
protocol: TCP
targetPort: 80
selector:
app: apache
type: ClusterIP
Next let's change Ingress resource test-ingress-2:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
namespace: test
name: test-ingress-2
annotations:
kubernetes.io/ingress.class: "service"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: nginx.test.com
http:
paths:
- path: /foo
backend:
serviceName: apache # <-------
servicePort: 80
Finally namespace test includes this list of resources:
# kubectl -n test get all
NAME READY STATUS RESTARTS AGE
pod/apache-cfdf8d79c-2m666 1/1 Running 0 13m
pod/apache-cfdf8d79c-d995s 1/1 Running 0 13m
pod/apache-cfdf8d79c-tq8d7 1/1 Running 0 13m
pod/nginx-65f88748fd-6w6fj 1/1 Running 0 45h
pod/nginx-65f88748fd-8fp7p 1/1 Running 0 45h
pod/nginx-65f88748fd-c7j29 1/1 Running 0 45h
pod/nginx-65f88748fd-frsbq 1/1 Running 0 46h
pod/nginx-65f88748fd-p4zvm 1/1 Running 0 45h
pod/nginx-65f88748fd-pd8gv 1/1 Running 0 45h
pod/nginx-65f88748fd-rkcjl 1/1 Running 0 45h
pod/nginx-65f88748fd-rn49k 1/1 Running 0 45h
pod/nginx-65f88748fd-w9dz8 1/1 Running 0 45h
pod/nginx-65f88748fd-xh42v 1/1 Running 0 45h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/apache ClusterIP 10.8.254.47 <none> 80/TCP 12m
service/nginx ClusterIP 10.8.254.218 <none> 80/TCP 12d
# kubectl -n test get ing
NAME HOSTS ADDRESS PORTS AGE
test-ingress-1 nginx.test.com 80 9m15s
test-ingress-2 nginx.test.com 80 54s
the ingreess-controller's nginx.conf shows two lists of upstreams
upstream test-apache-80 {
keepalive 32;
server 10.8.253.31:80 max_fails=0 fail_timeout=0;
server 10.8.252.208:80 max_fails=0 fail_timeout=0;
server 10.8.253.32:80 max_fails=0 fail_timeout=0;
}
upstream test-nginx-80 {
keepalive 32;
server 10.8.252.204:80 max_fails=0 fail_timeout=0;
server 10.8.252.205:80 max_fails=0 fail_timeout=0;
server 10.8.253.27:80 max_fails=0 fail_timeout=0;
server 10.8.253.25:80 max_fails=0 fail_timeout=0;
server 10.8.253.24:80 max_fails=0 fail_timeout=0;
server 10.8.252.206:80 max_fails=0 fail_timeout=0;
server 10.8.253.26:80 max_fails=0 fail_timeout=0;
server 10.8.252.203:80 max_fails=0 fail_timeout=0;
server 10.8.253.28:80 max_fails=0 fail_timeout=0;
server 10.8.252.201:80 max_fails=0 fail_timeout=0;
}
but it proxies requests only to one of them
## start server nginx.test.com
server {
server_name nginx.test.com ;
listen 80;
listen [::]:80;
set $proxy_upstream_name "-";
location ~* ^/foo\/?(?<baseuri>.*) {
set $namespace "test";
set $ingress_name "test-ingress-1"; <------
set $service_name "nginx";
set $service_port "80";
set $location_path "/foo";
...
rewrite /foo/(.*) /$1 break;
rewrite /foo / break;
proxy_pass http://test-nginx-80; <-------
proxy_redirect off;
# curl -H "Host: nginx.test.com" http://10.8.255.243:80/foo
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
Each of ingress resources appears there in accordance with the creation time. Deleting Ingress resource
ingress.networking.k8s.io "test-ingress-1" deleted
makes ingress-controller proxy traffic to Apache upstreams
# curl -H "Host: nginx.test.com" http://10.8.255.243:80/foo
<html><body><h1>It works!</h1></body></html>
to make it clear we can recreate Ingress resources in reverse order:
# kubectl -n test delete ing test-ingress-1 test-ingress-2
ingress.extensions "test-ingress-1" deleted
ingress.extensions "test-ingress-2" deleted
# kubectl -n test create -f /tmp/ing2.yaml
ingress.networking.k8s.io/test-ingress-2 created # Apache
# kubectl -n test create -f /tmp/ing1.yaml
ingress.networking.k8s.io/test-ingress-1 created # Nginx
# curl -H "Host: nginx.test.com" http://10.8.255.243:80/foo
<html><body><h1>It works!</h1></body></html> # Apache
One more note: There are some cases of reloading the ingress-controller's config, I mean when it reloads config and how it actually reloads config? But that's a totally different question...
If you mean Ingress objects with the same Controller: generally nothing interesting, most of the de-dup redundant routes.
If you mean two Controllers each pointed at a different (but identical) Ingress: both Controllers set themselves up with the requested routes. They are totally independent and don't know anything about each other.
Ingress controllers (whether it is nginx or haproxy or any other controller) work more or less in the same way. If an Ingress object is created or modified, they listen for that and for every the host/path in the ingress, they create a rule saying that if the request reaches the Ingress controller with that particular host/path, it should forward the traffic to the service that is configured for that path.
So, in your example, you are saying that you are adding multiple Ingress objects having the same host/path configs pointing to the same service. I would think that it is upto the Ingress controller implementation; how they want to handle duplicate Ingress rules. It doesn't make sense to add duplicate rules for the same hostname (doesn't matter whether it points to same service or different service, but in case of different backend service, it is a bigger problem/ambiguity).
But I would assume that even if an Ingress controller allows these kind of configs and programs multiple rules for the same host/path, it wont do multiplexing. Probably it would match the first rule and forward to the service behind it.
How it determines which service to route to? It is based on, on which hostname the traffic reached the Ingress controller.
You can easily check the behavior. Exec into the controller and see its conf file. For nginx ingress controller, the conf file will be in /etc/nginx/nginx.conf which will define the rules and actions.
We had a situation where such a configuration was needed in production. And this is how I saw it worked. When you create the second ingress with everything same except just the ingress name, it didn’t affect anything. Only after I deleted the first ingress, the second ingress that I created took over the control. So if the set up is indentical it doesn’t override or duplicate, looks like the first one that was created always has the prio.
