3

Earlier today I had increased my Docker desktop resources, but when ever since it restarted Kubernetes has not been able to complete its startup. Whenever I try to run a kubectl command, I get Unable to connect to the server: EOF in response.

I had thought that it started because I hadn't deleting a helm chart before adjusting the resource values in Settings, thus said resources having been assigned to the pods instead of the Kubernetes api server. But I have not been able to fix this issue.

This is what I have tried thus far:

  • Restarting Docker again
  • Reset Kubernetes
  • Reset Docker to factory settings
  • Deleting the VM in hyper-v and restarting Docker
  • Uninstalling and reinstalling Docker Desktop
  • Deleting the pki folder and restart Docker
  • Set the Environment Variable for KUBECONFIG
  • Deleting .kube/config and restart
  • Another clean reinstall of Docker Desktop

But Kubernetes does not complete its startup, so I still get Unable to connect to the server: EOF in response.

Is there anything I haven't tried yet?

shenyongo
  • 31
  • 1
  • 2
  • am I right that you are facing the issue on Windows? What's the Win version and Docker for desktop ver? It looks like networking issue. try restarting Win host itself – Nick Mar 24 '20 at 08:15
  • @Nick I have docker desktop version 2.0.0.3. Note that I can't upgrade beyond that otherwise it won't be able to pull my images from artifactory. My windows version is Windows 10 – shenyongo Mar 24 '20 at 13:14
  • @Nick while I was getting the error message still earlier this morning, the kubectl commands started to work again despite the docker settings saying that kubernetes was still starting... But when I restarted Docker again to increase my resources I get the same problem again. I definitely think kubernetes is taking a while to start up on my end for some reason – shenyongo Mar 24 '20 at 13:59
  • there is a way to check live logs https://docs.docker.com/docker-for-windows/troubleshoot/ may be there is some useful info ? – Nick Mar 24 '20 at 15:00
  • @Nick I had already tried looking at the logs and then doing a google search. This is the message in the logs that correlates with each kubectl command I try: `[10:16:43.986][ApiProxy ][Info ] time="2020-03-24T10:16:43-05:00" msg="Cannot list nodes: Get https://localhost:6445/api/v1/nodes: EOF"` – shenyongo Mar 24 '20 at 15:24
  • am i right that the `kubectl version` yeilds in a same error? – Nick Mar 24 '20 at 15:52
  • @Nick This is the message I get when I run kubectl version: `Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.11", GitCommit:"637c7e288581ee40ab4ca210618a89a555b6e7e9", GitTreeState:"clean", BuildDate:"2018-11-26T14:38:32Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"windows/amd64"} Unable to connect to the server: EOF` – shenyongo Mar 24 '20 at 15:53
  • @Nick also I had allowed Docker Desktop to show the kubernetes containers, and I noticed that the kube apiserver container, unlike the other kube containers, only stays up for a few seconds before being deleted (and it will repeat that from time to time) – shenyongo Mar 24 '20 at 17:00
  • @Nick Here are some of the logs I saw for the most recent kuve-apiserver container before it disappeared: `I0324 16:51:16.652790 1 server.go:724] external host was not specified, using 192.168.65.3` ... `F0324 16:51:37.135381 1 storage_decorator.go:57] Unable to create storage backend: config (&{ /registry [https://127.0.0.1:2379] /run/config/pki/apiserver-etcd-client.key /run/config/pki/apiserver-etcd-client.crt /run/config/pki/etcd/ca.crt true false 1000 0xc4201cce80 5m0s 1m0s}), err (dial tcp 127.0.0.1:2379: getsockopt: connection refused)` – shenyongo Mar 24 '20 at 17:01
  • have you managed to find the issue? – Nick May 22 '20 at 06:54
  • Solution for this is listed here: https://stackoverflow.com/questions/57711639/docker-for-windows-stuck-at-kubernetes-is-starting-after-updating-to-version-2 – Deepak Gangore Feb 17 '22 at 11:56

2 Answers2

3

I'll share that what solved this for me was Docker Desktop settings feature for "reset kubernetes cluster". I know that @shenyongo said that a "reset kubernetes" didn't work, and I suppose they mean this.

But for the sake of other readers who may find this, I had this same error message (with Docker Desktop on Windows 11, using wsl2), and the solution for me was indeed to do this:

  1. open the Settings page (in Docker Desktop--right-click on it in the status tray)
  2. then choose "Kubernetes" on the left
  3. then choose "reset kubernetes cluster"

Yes, that warns that "all stacks and kubernetes resources will be deleted", but as nothing else had worked for me (and I wasn't worried about losing much), I tried it, and it did the trick. In moments, all my k8s functionality was back to working.

As background, k8s had been working fine for me for some time. It was just that one day I found I was getting this error. I searched and searched and found lots of folks asking about it but not getting answers, let alone this answer. To be clear, like the OP here I had tried restarting Docker Desktop, restarting the host machine, even downloading and installing an available DD update (I was only a bit behind), and none of those worked. I didn't proceed to ALL the steps shenyongo did, as I thought I'd try this first, and the reset worked.

Hope that may help others. I realize some may fear losing something, but this helps stress the power of declarative vs imperative k8s configuration. It SHOULD be easy to recreate most everything if necessary. I realize it may not be so for everyone.

charlie arehart
  • 6,590
  • 3
  • 27
  • 25
0

As for me, what worked was ticking "Show system containers (advanced)" box, and then I clicked "APPLY AND RESTART".

Open Docker Desktop >> Settings >> Kubernetes >> Show system containers (advanced) >> APPLY AND RESTART

A screenshot of the Docker App

joshboy16
  • 1