3

When I run Kubernetes in Docker for Mac, the Kube API only seems to be accessible from a secure endpoint of https://localhost:6443/

With minikube I was able to use an insecure endpoint for Kube API like http://localhost:8080/

Is there any way to use an insecure endpoint for Kube API in Kubernetes in Docker for Mac?

frederix
  • 1,662
  • 5
  • 18
  • 29

1 Answers1

4

You may be running an old version of Kubernetes with minikube.

The default insecure port for the kube-apiserver is 8080, but that's disabled on the latest Kubernetes versions in the kube-apiserver with the flag: --insecure-port=0.

You can always delete that line from your /etc/kubernetes/manifests/kube-apiserver.yaml file.

You also need to add this option --insecure-bind-address=0.0.0.0 as per this.

Then restart the kube-apiserver.

Tip: Docker/Kubernetes runs on xhyve VM(s) on your Mac. So to modify the Kubernetes configs you'll have to connect to your xhyve VM(s). You can do it with something like this: screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty or screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

Adding more details: So the port 6443 is forwarded to the host using vpnkit. To make port 8080 available on the host you have to also expose that port with vpnkit. If you screen into the hyperkit vm you'll see that port mappings are defined in /var/vpnkit/port. There's a README file on that directory that you can follow to expose port 8080.

Rico
  • 58,485
  • 12
  • 111
  • 141
  • I can't find the `kube-apiserver.yaml` file on my Mac. Any idea how to change that setting in the docker for mac Kubernetes? – frederix Sep 20 '18 at 19:56
  • Looks like Kubernetes in Docker for Mac is [Not Configurable](https://docs.docker.com/docker-for-mac/kubernetes/#example-app) "_The Kubernetes server runs locally within your Docker instance, is not configurable, and is a single-node cluster._" Why even have it then...? – frederix Sep 20 '18 at 20:02
  • There might be a way. Added at the bottom of the answer – Rico Sep 20 '18 at 20:03
  • I have never used `screen` before and I'm struggling with it. Is the goal to screen into a terminal session and then find the `kube-apiserver.yaml` and modify it? Will I need to do this every time I boot up docker for mac? – frederix Sep 20 '18 at 20:17
  • That's the goal. You don't have to do every time you boot up docker for mac. It gets saved on the VM storage. – Rico Sep 20 '18 at 20:19
  • I used `screen` on that file twice because I didn't know what I was doing and now everything is garbled when I use `screen -r `. Not sure how to clear them or reset them. i have 4 of those sessions now when I use `screen -list`. – frederix Sep 20 '18 at 20:22
  • You can kill the screen processes: `kill -9 ` and start from the beginning – Rico Sep 20 '18 at 20:24
  • Had to restart docker. Then I was able to use `screen` and find the `kube-apiserver.yaml` file and modify it. Changed `--inescure-port` to 8080. Restarted docker, but still can't access Kube API on http://localhost:8080/api/v1/ – frederix Sep 20 '18 at 20:52
  • I'm getting *Connection refused* – frederix Sep 20 '18 at 20:55
  • Here is the config file: https://pastebin.com/raw/GpbW2ftM I checked that 8080 is still configured when docker restarts. – frederix Sep 20 '18 at 20:57
  • Try also adding this `--insecure-bind-address=0.0.0.0`. Did your kube-apiserver start at all? run a `docker ps` inside the VM to check. – Rico Sep 20 '18 at 21:07
  • Added `--insecure-bind-address=0.0.0.0`, restarted and still got a *Connection refused*. From `screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty` the docker command doesn't exist. `linuxkit-025000000001:~# docker ps -sh: docker: not found` – frederix Sep 20 '18 at 21:23
  • your kube-apiserver might not be running anymore. Try `ps -Af` .. see the processes running on the VM. – Rico Sep 20 '18 at 21:25
  • But still: `$ curl http://localhost:8080/api/v1 curl: (7) Failed to connect to localhost port 8080: Connection refused` – frederix Sep 20 '18 at 21:30
  • can you a curl within the VM? maybe it's not available. It could be a firewall rule somewhere. – Rico Sep 20 '18 at 21:33
  • Run this to check if something is listening on port 8080 `netstat -tunaple | grep 8080` – Rico Sep 20 '18 at 21:41
  • `linuxkit-025000000001:~# netstat -tunaple | grep 8080 tcp 0 0 :::8080 :::* LISTEN 3214/kube-apiserver` – frederix Sep 20 '18 at 21:46
  • And here is what I see that is listening on my mac: `$ lsof -i -n -P | grep TCP | grep LISTEN` `com.docke 63596 bfrede200 22u IPv4 0xa7cb42b3c642ffc9 0t0 TCP *:6443 (LISTEN)` `com.docke 63596 bfrede200 23u IPv6 0xa7cb42b3b29e6cc1 0t0 TCP [::1]:6443 (LISTEN)` – frederix Sep 20 '18 at 21:53
  • it's vpnkit. It's not forwarding port 8080 to the vm. – Rico Sep 20 '18 at 21:58
  • I'm not sure how to fix that port forwarding issue. – frederix Sep 20 '18 at 22:02
  • don't know either. I'll see if I can dig something up. What's the problem with minikube? – Rico Sep 20 '18 at 22:02
  • Nothing. I'm switching back to `minikube` because Docker for Mac's Kubernetes is clearly not useful for me. I thought it would be easier to use. I thought wrong :D – frederix Sep 20 '18 at 22:04
  • 2
    Thank you @Rico so much for this answer. Been struggling with this stuff for 2 days. – user674669 May 01 '19 at 17:25