0

I am attempting to use knative to self-host a simple FaaS platform from an on-premises Linux server. I have installed minikube successfully, and installed/configured knative using the knative operator.

Following the default selection in the linked guide, I have set up kourier as the networking layer and selected Magic DNS (sslip.io) setup. After running minikube tunnel, I can successfully serve demo apps like helloworld-python.

However, my "External IP" according to KNative is a private-ip:

kubectl --namespace knative-serving get service kourier

NAME      TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                      AGE
kourier   LoadBalancer   10.100.195.53   10.100.195.53   80:31140/TCP,443:31175/TCP   15h

(Note the IP starts with 10., this IP is not external to the machine.)

As such, the apps return sslip.io addresses containing this private IP: http://helloworld-python.default.10.100.195.53.sslip.io. As a result, I can query this service (e.g. via curl) just fine from the host server, but I can't access the service from any other machine. Likewise, if I attempt to configure any "Eventing" services, KNative produces webhooks with a private IP address, which obviously don't work since external services like GitHub don't resolve them.

So, why does Knative return a private IP in this setting? The KNative installation doesn't seem to give any indication that this will be the case, suggesting that

kubectl --namespace knative-serving get service kourier

might return a CNAME or an IP address and that we should "note this for DNS configuration later" (except if using Magic DNS). Should it be returning a private IP?

  • If so, what is the procedure for exposing services? Do I need to run some additional configuration to expose a public IP?

  • If not, is some additional per-configuration needed so that it uses the machine's public IP? (or at least an IP I can bind my own reverse proxy to?)

  • Is Kourier designed to only do private IPs? (The docs are pretty terse about why one should chose kourier vs istio or contour for networking, merely saying that istio is the default but new users should chose kourier if they are unsure??)

As a side-note/ background context, my server has a fixed IP address, I use caddy (in container on the same docker network as minikube) to provide https domain names to services. Also, outside of knative, if I just follow standard minikube guide to make a service as NodePort or LoadBalancer, I can easily take the port that kubectl get svc shows, and expose the service in caddy by pointing to said port on the minikube container, something like this in the Caddyfile:

minikube.app.mydomain.com {
  reverse_proxy minikube:32637 {
    header_up Host {host}
  } 
}

This is just to confirm that in vanilla minikube I have no trouble with ingress configuration. KNative clearly has this extra layer of networking by which it's generating URLs for applications though and that has be flummoxed. I can't just replace the `minikube:32637 with the private IP addresses KNative returns)

cboettig
  • 12,377
  • 13
  • 70
  • 113

1 Answers1

1

Knative uses the HTTP Host header to share a single IP address across many services (sorta like what you're doing with caddy...).

If you want to use caddy to route these requests, you'll need to rewrite the Host header using the header_up directive. You may be able to use the replacement form and/or change the Knative domain prefix to make this easier.

E. Anderson
  • 3,405
  • 1
  • 16
  • 19
  • Thanks. Yes I see KNative mapping the IP address across different services with subdomains, that makes total sense. The difficulty I have is that it has chosen to map a Private IP address, 10.100.195.53, so these services are not accessible. I don't understand the mechanism by which I expose that. In particular, I can't even tell if I am supposed to roll with that private IP by redirecting external requests to it, or alter configuration somehow to use a public IP. Caddy passes through the headers as you know. I can add `header_up {host}` to my ex above but does not address the issue. – cboettig Sep 04 '22 at 18:52
  • The "external ip" detected by Knative is simply using the `minikube tunnel` IP address. – E. Anderson Sep 04 '22 at 20:25
  • Try something like ```minikube.app.mydomain.com { reverse_proxy 10.100.195.53:80 { header_up Host helloworld-python.default.10.100.195.53.sslip.io } }``` – E. Anderson Sep 04 '22 at 20:27
  • oh hmm I see what you mean now about header_up, clever! It still doesn't like that though. Good to know what is responsible for selecting the external IP, perhaps I need to look at configuring minikube tunnel instead of adjusting something in caddy or knative(?) – cboettig Sep 05 '22 at 02:38