Kubernates Nginx-Ingress rule not working

Question

I installed the ingress controller using the following command:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.5.1/deploy/static/provider/cloud/deploy.yaml

And the result of kubectl get pods --namespace=ingress-nginx is:

NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-x4mss        0/1     Completed   0          28m
ingress-nginx-admission-patch-jn9cz         0/1     Completed   1          28m
ingress-nginx-controller-8574b6d7c9-k4jbj   1/1     Running     0          28m

For kubectl get service ingress-nginx-controller --namespace=ingress-nginx I get:

NAME                       TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller   LoadBalancer   10.106.134.128   localhost     80:32294/TCP,443:30997/TCP   30m

As for my deployment and service I have the following:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: app
  name: app
  namespace: namespace
spec:
  replicas: 1
  selector:
    matchLabels:
      app: app
  template:
      labels:
        app: app
    spec:
      containers:
        - image: image
          name: app
          ports:
            - containerPort: 5000
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: app-service
  namespace: namespace
spec:
 type: ClusterIP
 selector:
   app: app
  ports:
    - name: app-service
      port: 5000
      targetPort: 5000

My Ingress is as follows:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
  namespace: namespace
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: com.host.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 5000

My pod and service are both running fine. The result of running kubectl describe pod command is:

Name:             app-6b9f7fc47b-sh6nc
Namespace:        namespace
Priority:         0
Service Account:  default
Node:             docker-desktop/192.168.65.4
Start Time:       Wed, 30 Nov 2022 16:22:04 -0500
Labels:           app=app
                  pod-template-hash=6b9f7fc47b
Status:           Running
IP:               10.1.0.237
IPs:
  IP:           10.1.0.237
Controlled By:  ReplicaSet/app-6b9f7fc47b
Containers:
  app:
    Container ID:   docker://ba77235d044c24b0f1391c56a2e8653a598a5c130ea4d15ff3b41cd96659fd4a
    Image:          image
    Image ID:       docker://sha256:912cb58ab1c3f2dd628c0b7db4d7f9ac6df4efbe4fcb86979b6a84614db8a675
    Port:           5000/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Wed, 30 Nov 2022 16:22:05 -0500
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-8pmjz (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-8pmjz:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  29m   default-scheduler  Successfully assigned namespace/app-6b9f7fc47b-sh6nc to docker-desktop
  Normal  Pulled     29m   kubelet            Container image "image" already present on machine
  Normal  Created    29m   kubelet            Created container app
  Normal  Started    29m   kubelet            Started container app

Running the following command kubectl get ingress --all-namespaces yields:

NAMESPACE           NAME             CLASS   HOSTS               ADDRESS   PORTS   AGE
namespace          ingress           nginx   com.host.com                   80      7s

I have tried using different ports, changing the controller, using a load balance type instead of cluster ip and yet nothing works when it comes to trying to make the ingress rule work. I have set the ingress-controller external ip as com.host.com in my hosts file as well. Furthermore, I am using docker-desktop as my node, however, I'm having this issue on minikube as well. Any help is appreciated.

What's the error? Do you get connection timeout? Or not found? Is there something in the logs of the nginx ingress controller? — user2311578, Nov 30 '22 at 21:31
@user2311578 I get a site can't be reached error in the browser. In the nginx controller logs, I see the following errors `Error obtaining Endpoints for Service "namespace/app-service" no object matching key "namespace/app-service" in local store. Service "namespace/app-service" does not have any active Endpoint.` But I believe those may be old errors. — Interlunar, Nov 30 '22 at 21:44
What is the external IP of your load balancer? Is it reachable from your machine? Can you use curl to check? Newer browsers might tend to force HTTPS. `curl -v http:///` You don't need the host in your ingress rule. — user2311578, Nov 30 '22 at 21:54
Also look if there is info on `kubectl -n namespace describe ingress ingress`. It's strange that there is no ADDRESS. — user2311578, Nov 30 '22 at 21:57
@user2311578 I get ` Name: ingress Labels: Namespace: namespace Address: localhost Ingress Class: nginx Default backend: Rules: Host Path Backends ---- ---- -------- com.host.com / app:3000 (10.1.0.237:3000) Annotations: Events: ` — Interlunar, Nov 30 '22 at 22:02
As for the curl request, I get The remote name could not be resolved: 'com.host.com' for com.host.com, I get a 404 if I do curl -v 127.0.0.1:80 and I get a respond if the service pod is of type loadbalancer. — Interlunar, Nov 30 '22 at 22:05
Did you remove the host from the ingress? If not, you can add the host header to curl: `curl -v -H "host: com.host.com" http://127.0.0.1:80` — user2311578, Nov 30 '22 at 22:08
@user2311578 I didn't remove the host from the ingress. But I get this error with the command Invoke-WebRequest : Cannot bind parameter 'Headers'. Cannot convert the "host: com.alfredgui.com" value of type "System.String" to type "System.Collections.IDictionary". At line:1 char:12 + curl -v -H "host: com.alfredgui.com" http://127.0.0.1:80 — Interlunar, Nov 30 '22 at 22:16
You're on windows and not using the original curl. See here: https://stackoverflow.com/questions/45183355/invoke-webrequest-cannot-bind-parameter-headers Anyways, seems the problem is with name resolution (your hosts file entry is not taken). I've updated my answer. — user2311578, Nov 30 '22 at 22:22
@user2311578 I think you're absolutely right. Using curl.exe with the header command worked. — Interlunar, Nov 30 '22 at 22:29

user2311578 · Accepted Answer · 2022-11-30T22:14:04.323

0

Your container's containerPort is 3000 while your service targetPort is 5000. Make sure that your service' targetPort, your container's containerPort and last but not least the listening port of your app are all the same.

Update:

So I guess it turned out that the problem is with name resolution.

com.host.com is not resolvable on your machine.

Using the localhost or 127.0.0.1 didn't match the host from the ingress rule -> that's why the default 404 handler was chosen.

So either fix name resolution or remove the host from the ingress rule.

edited Nov 30 '22 at 22:14

answered Nov 30 '22 at 21:19

user2311578

798
3
7

Sorry, this was a typo in the question. They are the same. I will update the question to reflect this. Sorry. – Interlunar Nov 30 '22 at 21:21

Blender Fox · Answer 2 · 2022-11-30T22:06:29.263

0

Bear in mind that the ingress controller will only update the IP of the ingress if the mapped service is up and ready. Your ingress shows that it's using app-service on port 5000 as its backend service, but your question does not show a listing of the pods on the namespace namespace where it appears your application pods are. Please could you add that to the question -- it is possible that either your pods aren't coming up successfully, or they're listening on a different port to 5000

UPDATE

Please also try the following:

You're specifying that the container uses port 5000. But is it actually using 5000? try:

kubectl exec -it -n namespace app-6b9f7fc47b-sh6nc -- bash (or sh depending on what the default shell is for your app)

and then try curl localhost:5000 to see if it responds.

Check the ingress controller logs:

kubectl logs -f -n ingress-nginx and see if there's any log messages that might help you identify what's going on.

edited Nov 30 '22 at 22:06

answered Nov 30 '22 at 21:39

Blender Fox

4,442
2
17
30

I added the information to the question. – Interlunar Nov 30 '22 at 21:57
Updated answer with more checks – Blender Fox Nov 30 '22 at 22:06
Response to update: I get a response with curl localhost:5000. As for the logs I do get `Error obtaining Endpoints for Service "namespace/app-service" no object matching key "namespace/app-service" in local store. Service "namespace/app-service" does not have any active Endpoint.` But I believe this may be an old error. – Interlunar Nov 30 '22 at 22:19

Kubernates Nginx-Ingress rule not working

2 Answers2