0

Kubernetes dedicated cockroachdb node - accessing admin ui via traefik ingress controller fails - page isn't redirecting properly

I have a dedicated kubernetes node running cockroachdb. The pods get scheduled and everything is setup. I want to access the admin UI from a subdomain like so: cockroachdb.hostname.com. I have done this with traefik dashboard and ceph dashboard so I know my ingress setup is working. I even have cert-manager running to have https enabled. I get the error from the browser that the page is not redirecting properly.

Do I have to specify the host name somewhere special?

I have tried adding this with no success: --http-host cockroachdb.hostname.com

This dedicated node has its own public ip which is not mapped to hostname.com. I think I need to change a setting in cockroachdb, but I don't know which because I am new to it.

Does anyone know how to publish admin UI via an ingress?

EDIT01: Added ingress and service config files

Ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cockroachdb-public
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.rule.type: PathPrefixStrip
    certmanager.k8s.io/issuer: "letsencrypt-prod"
    certmanager.k8s.io/acme-challenge-type: http01
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/ssl-temporary-redirect: "true"
    ingress.kubernetes.io/ssl-host: "cockroachdb.hostname.com"
    traefik.frontend.rule: "Host:cockroachdb.hostname.com,www.cockroachdb.hostname.com"
    traefik.frontend.redirect.regex: "^https://www.cockroachdb.hostname.com(.*)"
    traefik.frontend.redirect.replacement: "https://cockroachdb.hostname.com/$1"
spec:
  rules:
  - host: cockroachdb.hostname.com
    http:
      paths:
      - path: /
        backend:
          serviceName: cockroachdb-public
          servicePort: http
  - host: www.cockroachdb.hostname.com
    http:
      paths:
      - path: /
        backend:
          serviceName: cockroachdb-public
          servicePort: http
  tls:
  - hosts:
    - cockroachdb.hostname.com
    - www.cockroachdb.hostname.com
    secretName: cockroachdb-secret

Serice:

apiVersion: v1
kind: Service
metadata:
  # This service is meant to be used by clients of the database. It exposes a ClusterIP that will
  # automatically load balance connections to the different database pods.
  name: cockroachdb-public
  labels:
    app: cockroachdb
spec:
  ports:
  # The main port, served by gRPC, serves Postgres-flavor SQL, internode
  # traffic and the cli.
  - port: 26257
    targetPort: 26257
    name: grpc
  # The secondary port serves the UI as well as health and debug endpoints.
  - port: 8080
    targetPort: 8080
    name: http
  selector:
    app: cockroachdb

EDIT02:

I can access the Admin UI page now but only by going over the external ip address of the server with port 8080. I think I need to tell my server that its ip address is mapped to the correct sub domain?

EDIT03:

On both scheduled traefik-ingress pods the following logs are created: time="2019-04-29T04:31:42Z" level=error msg="Service not found for default/cockroachdb-public"

krjw
  • 4,070
  • 1
  • 24
  • 49
  • Please share your manifest files, that hold the Ingress/Service resource definitions. I understand, that you can safely reach cockroachdb UI with e.g. kubectl port-forward – Nepomucen Apr 30 '19 at 14:32
  • @Nepomucen I added the resource definitions. The cockroachdb one is the standard taken from the documentation. I think it has to do with the ports and the fact that i have a dedicated node with its own external ip adress and different hostname. – krjw Apr 30 '19 at 14:39
  • What is the error you are getting? When you get the logs for your Traefik pod, what do you see for the failed connection attempts? – cookiedough Apr 30 '19 at 14:42
  • oh yeah something is wrong ... `time="2019-04-29T07:00:35Z" level=error msg="Service not found for default/cockroachdb-public"` this is coming in infinite loop – krjw Apr 30 '19 at 14:44
  • Do you see your service when you run `kubectl get svc`? – cookiedough Apr 30 '19 at 14:48
  • yet its there. `cockroachdb-public ClusterIP 10.96.235.225 26257/TCP,8080/TCP 76m` – krjw Apr 30 '19 at 14:49
  • @cookiedough this doesn't even stop after deleting the ingress – krjw Apr 30 '19 at 15:00
  • What do you get when you run `kubectl port-forward svc/cockroachdb-public 8080:8080`? Can you access it on localhost:8080? – cookiedough Apr 30 '19 at 15:02
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/192640/discussion-between-krjw-and-cookiedough). – krjw Apr 30 '19 at 15:02

1 Answers1

1

Your referencing looks good on the ingress side. You are using quite a few redirects, unless you really know what each one is accomplishing, don't use them, you might end up in an infinite loop of redirects.

You can take a look at the following logs and methods to debug:

Run kubectl logs <traefik pod> and see the last batch of logs.

Run kubectl get service, and from what I hear, this is likely your main issue. Make sure your service exists in the default namespace.

Run kubectl port-forward svc/cockroachdb-public 8080:8080 and try connecting to it through localhost:8080 and see terminal for potential error messages.

Run kubectl describe ingress cockroachdb-public and look at the events, this should give you something to work with.

Try accessing the service from another pod you have running ping cockroachdb-public.default.svc.cluster.local and see if it resolves the IP address.

Take a look at your clusterrolebindings and serviceaccount, it might be limited and not have permission to list services in the default namespace: kubectl create clusterrolebinding default-admin --clusterrole cluster-admin --serviceaccount=default:default

cookiedough
  • 3,552
  • 2
  • 26
  • 51