37

What is the equivalent command for minikube delete in docker-for-desktop on OSX

As I understand, minikube creates a VM to host its kubernetes cluster but I do not understand how docker-for-desktop is managing this on OSX.

2240
  • 1,547
  • 2
  • 12
  • 30
John Smith
  • 433
  • 1
  • 4
  • 9

6 Answers6

42

Tear down Kubernetes in Docker for OS X is quite an easy task.

Go to Preferences, open Reset tab, and click Reset Kubernetes cluster.

enter image description here

All object that have been created with Kubectl before that will be deleted.

You can also reset docker VM image (Reset disk image) and all settings (Reset to factory defaults) or even uninstall Docker.

VAS
  • 8,538
  • 1
  • 28
  • 39
  • 2
    On Docker for Mac `18.06.1-ce`, there's no Reset Kubernetes cluster button. However, disabling Kubernetes and hitting Apply on the Kubernetes tab worked. – RichVel Jan 02 '19 at 08:31
  • 4
    Any way to do this from a command line? It sucks to go into UI each time. – demisx Apr 20 '19 at 13:51
  • This is actually the same for docker for desktop on windows – Dominik Jan 09 '20 at 17:14
  • 1
    Thanks for this. It appears it's been moved to the "Kubernetes" tab in preferences, but there's still a Reset button. – Matt Browne Mar 06 '20 at 21:47
  • Is there a standard way to remove the kubectl installation? – Kevin McDonough Nov 20 '21 at 01:07
  • It depend on how you install it. There are several ways to get it and it should be removed differently. It would be better to ask separate question on Stack with more details. – VAS Nov 20 '21 at 08:51
25

In recent Docker Edge versions for Mac ( 2.1.7 ) Preferences design has been changed. Now you can reset Kubernetes cluster and other docker aspects by switching to the bug plane in the top right of Preferences window:

Note: You are able to reset Kubernetes cluster only if it's enabled. If you uncheck "Enable Kubernetes" checkbox, "Reset Kubernetes cluster" button becomes inactive.

enter image description here

For convenience "Reset Kubernetes cluster" is also present on the Kubernetes tab in the main Preferences plane:

enter image description here

VAS
  • 8,538
  • 1
  • 28
  • 39
6

To reset Docker-desktop Kubernetes cluster using command line, put the following content to a file (dd-reset.sh) and mark it executable ( chmod a+x dd-reset.sh )

#!/bin/bash

dr='docker run -it --rm --privileged --pid=host debian nsenter -t 1 -m -u -n -i'

${dr} sh -c 'export PATH=$PATH:/containers/services/docker/rootfs/usr/bin:/containers/services/docker/rootfs/usr/local/bin:/var/lib/kube-binary-cache/ && \
if [ ! -e /var/run/docker.sock ] ; then ln -s /containers/services/docker/rootfs/var/run/docker.sock /var/run/docker.sock ; fi && \
kube-reset.sh'

sleep 3

echo "cluster resetted. restarting docker-desktop..."
osascript -e 'quit app "Docker"'

open --background -a Docker
echo "docker-desktop started. Wait 3-5 mins for kubernetes to start."

Explanation:

This method uses internal scripts from Docker-desktop VM. To make it work, some preparation of user environment is required.

I wasn't able to start Kubernetes cluster using kube-start.sh script from inside the VM, so I've used MacOS commands to restart Docker application instead.

This method works even if your Kubernetes cluster is not enabled in Docker preferences at the moment, but it's required to enable Kubernetes at least once to use the script.

It was tested on Docker Edge for MacOS v2.2.2.0 (43066)
There is no guarantee that it will be compatible with earlier or later versions.

This version of Docker uses kubeadm to initialize Kubernetes cluster. Scripts are located in the folder /containers/services/docker/rootfs/usr/bin:

  • kube-pull.sh (brings kubernetes binaries to VM)
  • kube-reset.sh (runs kube-stop.sh and do kubeadm reset + some rm stuff)
  • kube-restart.sh (runs kube-stop.sh and kube-start.sh)
  • kube-start.sh (runs kube-pull.sh and kubelet.sh)
  • kube-stop.sh (kills kubelet and kube-apiserver processes, and all k8s containers)
  • kubeadm-init.sh (initializes Kubernetes cluster)
  • kubelet.sh (runs kubeadm-init.sh and starts kubelet binary)

Cluster configuration is located in the file /containers/services/docker/lower/etc/kubeadm/kubeadm.yaml

Resources used:

VAS
  • 8,538
  • 1
  • 28
  • 39
  • Nice! (the UI will still be stuck in "Kubernetes is starting" if Docker is in that state) – Matt Oct 18 '20 at 21:48
  • some updates about connecting to docker-desktop vm: https://gist.github.com/BretFisher/5e1a0c7bcca4c735e716abf62afad389 – VAS Jul 01 '21 at 12:22
4

It's really under the hood in the code. Docker for Mac uses these components: Hyperkit, VPNkit and DataKit

Kubernetes runs in the same Hyperkit VM created for docker and the kube-apiserver is exposed.

You can connect to the VM with this:

screen ~/Library/Containers/com.docker.docker/Data/vms/0/tty

Then you can see all the Kubernetes processes in the VM:

linuxkit-025000000001:~# ps -Af | grep kube
 1251 root      0:00 /usr/bin/logwrite -n kubelet /usr/bin/kubelet.sh
 1288 root      0:51 kubelet --kubeconfig=/etc/kubernetes/kubelet.conf --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true --cluster-dns=10.96.0.10 --cluster-domain=cluster.local --cgroups-per-qos=false --enforce-node-allocatable= --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --cadvisor-port=0 --kube-reserved-cgroup=podruntime --system-reserved-cgroup=systemreserved --cgroup-root=kubepods --hostname-override=docker-for-desktop --fail-swap-on=false
 3564 root      0:26 kube-scheduler --address=127.0.0.1 --leader-elect=true --kubeconfig=/etc/kubernetes/scheduler.conf
 3616 root      1:45 kube-controller-manager --cluster-signing-key-file=/run/config/pki/ca.key --address=127.0.0.1 --root-ca-file=/run/config/pki/ca.crt --service-account-private-key-file=/run/config/pki/sa.key --kubeconfig=/etc/kubernetes/controller-manager.conf --cluster-signing-cert-file=/run/config/pki/ca.crt --leader-elect=true --use-service-account-credentials=true --controllers=*,bootstrapsigner,tokencleaner
 3644 root      1:59 kube-apiserver --admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,NodeRestriction,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --requestheader-username-headers=X-Remote-User --requestheader-group-headers=X-Remote-Group --service-account-key-file=/run/config/pki/sa.pub --secure-port=6443 --insecure-port=8080 --insecure-bind-address=0.0.0.0 --requestheader-client-ca-file=/run/config/pki/front-proxy-ca.crt --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --requestheader-extra-headers-prefix=X-Remote-Extra- --advertise-address=192.168.65.3 --service-cluster-ip-range=10.96.0.0/12 --tls-private-key-file=/run/config/pki/apiserver.key --enable-bootstrap-token-auth=true --requestheader-allowed-names=front-proxy-client --tls-cert-file=/run/config/pki/apiserver.crt --proxy-client-key-file=/run/config/pki/front-proxy-client.key --proxy-client-cert-file=/run/config/pki/front-proxy-client.crt --allow-privileged=true --client-ca-file=/run/config/pki/ca.crt --kubelet-client-certificate=/run/config/pki/apiserver-kubelet-client.crt --kubelet-client-key=/run/config/pki/apiserver-kubelet-client.key --authorization-mode=Node,RBAC --etcd-servers=https://127.0.0.1:2379 --etcd-cafile=/run/config/pki/etcd/ca.crt --etcd-certfile=/run/config/pki/apiserver-etcd-client.crt --etcd-keyfile=/run/config/pki/apiserver-etcd-client.key
 3966 root      0:01 /kube-dns --domain=cluster.local. --dns-port=10053 --config-dir=/kube-dns-config --v=2
 4190 root      0:05 /usr/local/bin/kube-proxy --config=/var/lib/kube-proxy/config.conf
 4216 65534     0:03 /sidecar --v=2 --logtostderr --probe=kubedns,127.0.0.1:10053,kubernetes.default.svc.cluster.local,5,SRV --probe=dnsmasq,127.0.0.1:53,kubernetes.default.svc.cluster.local,5,SRV
 4606 root      0:00 /compose-controller --kubeconfig  --reconciliation-interval 30s
 4905 root      0:01 /api-server --kubeconfig  --authentication-kubeconfig  --authorization-kubeconfig  --etcd-servers=https://127.0.0.1:2379 --etcd-cafile=/etc/docker-compose/etcd/ca.crt --etcd-certfile=/etc/docker-compose/etcd/client.crt --etcd-keyfile=/etc/docker-compose/etcd/client.key --secure-port=9443 --tls-ca-file=/etc/docker-compose/tls/ca.crt --tls-cert-file=/etc/docker-compose/tls/server.crt --tls-private-key-file=/etc/docker-compose/tls/server.key

So if you uncheck the following box (unclear from the docs what command it uses):

docker

You can see that the processes are removed:

linuxkit-025000000001:~# [ 6616.856404] cni0: port 2(veth5f6c8b28) entered disabled state
[ 6616.860520] device veth5f6c8b28 left promiscuous mode
[ 6616.861125] cni0: port 2(veth5f6c8b28) entered disabled state

linuxkit-025000000001:~#
linuxkit-025000000001:~# [ 6626.816763] cni0: port 1(veth87e77142) entered disabled state
[ 6626.822748] device veth87e77142 left promiscuous mode
[ 6626.823329] cni0: port 1(veth87e77142) entered disabled state

linuxkit-025000000001:~# ps -Af | grep kube
linuxkit-025000000001:~#
Rico
  • 58,485
  • 12
  • 111
  • 141
  • 1
    yeah I did that but when you enable it again then the cluster is still there – John Smith Oct 18 '18 at 17:39
  • This worked for me on Docker for Mac `18.06.1-ce` - you must hit Apply for the change to take effect. – RichVel Jan 02 '19 at 08:32
  • 2
    @Rico is there a single line to reset the kubernetes cluster on docker for mac programatically, instead of through the GUI? – Andrew Allbright Sep 09 '19 at 20:59
  • There must be either an api endpoint and/or a command line, however it doesn't seem to be published. You might have to look at the source code. – Rico Sep 11 '19 at 22:22
3

On docker desktop version 3.5.2 (engine version 20.10.7), the reset button has been moved inside the docker preferences.

You can get there by following the below steps:

  1. Click on the docker icon in the menu bar and choose 'Preferences'. enter image description here
  2. Go to the Kubernetes tab. enter image description here
  3. Click on the Reset Kubernetes CLuster button. This is the red color button.

This will delete all pods and reset the kubernetes. You can execute the docker ps command at terminal to verify that there are no containers running.enter image description here

matrix
  • 3,000
  • 3
  • 24
  • 35
0

in my cased docker kubernetes stopped working with docker desktop v4.17.0 because the coredns pods were not able to pull the specified image.

execute kubectl get all --all-namespaces on the local cluster, if coredns pods are failing to start, try to apply kubectl patch deployment -n kube-system coredns --patch-file docker-local-core-dns-image-patch.yaml

docker-local-core-dns-image-patch.yaml file content updates the image to pull:

spec:
  template:
    spec:
      containers:
      - name: coredns
        image: registry.k8s.io/coredns/coredns:v1.9.3
Bauss
  • 51
  • 1
  • 3