NGINX Ingress to Microk8s Bare metal cluster not working as expected

Question

First a little background:

We currently have several websites and services hosted on a Plesk server and I am setting up a bare-metal development server to provide an area where we can test updates, etc. before going to production. I am using a 3 node kubernetes cluster running microk8s on Ubunutu 20.04.01. The services we host are pretty diverse: we have a couple Moodle sites, a few Wordpress sites, a site running limesurvery, an instance of Mantis bugtracker, and a few more. I have successfully gotten most of the sites containerized and running on k8s. I can also access each individual site either through a NodePort or a MetalLB load balancer.

However, I'd really like to use the NGINX Ingress Controller on top of the load balancer so that I can have a consistent way to access the sites without using a bunch of IP addresses (or in the NodePort case, ports that change). No matter what I've done, I cannot seem to get the Ingress to do what I want. I simply want to do the following: http://<LB IP Address>/bugtracker to access the Mantis Bug Tracker site http://<LB IP Address>/moodle1 to access one of the Moodle sites http://<LB IP Address>/limesurvey to access the limesurvey, etc. I seem to be able to get to the main page of the site (e.g. index.html, index.php, etc.), but any references from there on do not work - i.e. they give path not found errors or 404 errors.

Here's a sample of my Ingress file:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - http:
      paths:
      - path: /limesurvey(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: limesurvey-svc
            port:
              number: 80
      - path: /moodle(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: moodle-svc
            port:
              number: 8080

This ingress does not work (I get the 404s). However, if I only have one path in the file and just use '/' it works (but I can only use it for one service):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: limesurvey-svc
            port:
              number: 80

I think what I need is for the path (limesurvey) to continue to be pre-pended onto each URL request, but I cannot seem to make this happen. The reason I think this is because when I navigate to http://<LB IP Address>/limesurvey and click on a survey, I get a 404 error at http://<LB IP Address>/index.php/<survey number>. However, if I manually change the URL in my browser to http://<LB IP Address>/limesurvey/index.php/<survey number> it will access the survey (but still have 404s with supporting assets).

Am I trying to do something outside of what the Ingress controller was designed for? I feel like I should be able to use the rewrite-target to accomplish this but I'm missing something critical.

I appreciate any help.

Answer 1

Some applications relay on static content served from different URL webserver locations and moreover do internal path routing (e.g. you hit "/" path but get served "/admin" section immediately).

In such cases creation of right Ingress rules gets more tricky, and requires you to better understand behavior and constructs of your web application, to predict all possible URL path locations that user may visit (these forced by app internal redirects too), and these sourced by html code as well.

Seems like your case with limesurvey app falls into that category:

Why do I think that?
Just try to open limesurvey public demo (https://demo.limesurvey.org/) and inspect site content.

You will learn that main page is using a lot of static files (e.g. css, javascrpt files), referenced from absolute path starting with: /tmp/assets/...

<link rel="stylesheet" type="text/css" href="/tmp/assets/2d523ae6/survey.css" />

of course variants of different path locations can be matched with single smart reg-ex pattern, to avoid creation of dozen of individual ingress rules (what you tried).

What's the issue?

rules:
  - http:
      paths:
      - path: /limesurvey(/|$)(.*) <---- it won't match "/tmp/assets/..." location
        pathType: Prefix
        backend:
          service:
            name: limesurvey-svc
            port:
              number: 80

Please try to create additional Ingress rule to support static file location (watch out, I'm using old syntax of Ingress resource, adjust it to your needs):

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$1
  name: ingress-limesurvey-static
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: limesurvey-svc
          servicePort: 80
        path: /?(.*)

Best Solution (in my opinion)

You should define custom public URL within your application directly. Detailed information can be found in Advanced path settings, see publicurl option.

This way you wouldn't need to define internal reference for static files, however it should be done during installation.