Running multiple applications parallel on single Ingress Controller (e.g. JupyterLab/Elyra + Kubeflow Pipelines UI)

Question

I've installed Elyra (which is JupyerLab + some AI centric extensions including a visual editor for creating Kubeflow Pipelines) and KubeFlow Pipelines in parallel on a KIND (Kubernetes In Docker) cluster.

First I've installed Kubeflow Pipelines as documented here

export PIPELINE_VERSION=1.4.1
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION"
kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io
kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-pns?ref=$PIPELINE_VERSION"

Then, I've installed the NGINX Ingress Controller as documented here

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml

Finally, I've installed Elyra/JupyterLab using the following steps

So all UIs I need have the respective services already created:

k get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
elyra-ai NodePort 10.96.84.31 8888:32111/TCP 94m
kubernetes ClusterIP 10.96.0.1 443/TCP 105m

k get svc -n kubeflow |grep ml-pipeline-ui
ml-pipeline-ui ClusterIP 10.96.132.233 80/TCP 105m

Therefore I've created an ingress.yaml to enable JupyterLab/Elyra to be accessed.

As you can see I'm adding many path entries like login, elyra, git, lsp, lab, kernelspecs, static, ...

So now I'm stuck in two dimensions.

I don't want to add path entries for each and every JupyterLab extension I'm installing
I can't find a way to also make the Kubeflow Pipelines UI accessible from outside the cluster

The Elyra Pipeline Editor can access the Kubeflow Pipelines Endpoint via ClusterIP though but would be nice have access to the Kubeflow Pipelines UI as well...

score 1 · Answer 1 · answered Apr 14 '21 at 13:30

The easiest way is to use virtual hosts. So either in a DNS or in your /etc/hosts file add for example:

127.0.0.1 elyra.local.host
127.0.0.1 kubeflow.local.host

Then the two service.yaml become straightforward, for Elyra/JupyteLab:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: elyra-ingress
spec:
  rules:
  - host: "elyra.local.host"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: elyra-ai
            port:
              number: 8888

For Kubeflow Pipelines:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kfp-ingress
spec:
  rules:
  - host: "kubeflow.local.host"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: ml-pipeline-ui
            port:
              number: 80

Running multiple applications parallel on single Ingress Controller (e.g. JupyterLab/Elyra + Kubeflow Pipelines UI)

1 Answers1