2

I have several Windows servers available and would like to setup a Kubernetes cluster on them. Is there some tool or a step by step instruction how to do so?

What I tried so far is to install DockerDesktop and enable its Kubernetes feature. That gives me a single node Cluster. However, adding additional nodes to that Docker-Kubernetes Cluster (from different Windows hosts) does not seem to be possible: Docker desktop kubernetes add node

Should I first create a Docker Swarm and could then run Kubernetes on that Swarm? Or are there other strategies?

I guess that I need to open some ports in the Windows Firewall Settings of the hosts? And map those ports to some Docker containers in which Kubernetes is will be installed? What ports?

Is there some program that I could install on each Windows host and that would help me with setting up a network with multiple hosts and connecting the Kubernetes nodes running inside Docker containers? Like a "kubeadm for Windows"?

Would be great if you could give me some hint on the right direction.

Edit:
Related info about installing kubeadm inside Docker container:
https://github.com/kubernetes/kubernetes/issues/35712
https://github.com/kubernetes/kubeadm/issues/17

Related question about Minikube:
Adding nodes to a Windows Minikube Kubernetes Installation - How?

Info on kind (kubernetes in docker) multi-node cluster:
https://dotnetninja.net/2021/03/running-a-multi-node-kubernetes-cluster-on-windows-with-kind/ (Creates multi-node kubernetes cluster on single windows host)
Also see:

Stefan
  • 10,010
  • 7
  • 61
  • 117

1 Answers1

1

You can always refer to the official kubernetes documentation which is the right source for the information.

This is the correct way to manage this question.

Based on Adding Windows nodes, you need to have two prerequisites:

  • Obtain a Windows Server 2019 license (or higher) in order to configure the Windows node that hosts Windows containers. If you are using VXLAN/Overlay networking you must have also have KB4489899 installed.

  • A Linux-based Kubernetes kubeadm cluster in which you have access to the control plane (see Creating a single control-plane cluster with kubeadm).

Second point is especially important since all control plane components are supposed to be run on linux systems (I guess you can run a Linux VM on one of the servers to host a control plane components on it, but networking will be much more complicated).

And once you have a proper running control plane, there's a kubeadm for windows to proper join Windows nodes to the kubernetes cluster. As well as a documentation on how to upgrade windows nodes.

For firewall and which ports should be open check ports and protocols.

For worker node (which will be windows nodes):

Protocol    Direction   Port Range    Purpose            Used By
TCP         Inbound     10250         Kubelet API        Self, Control plane
TCP         Inbound     30000-32767   NodePort Services  All

Another option can be running windows nodes in cloud managed kuberneres, for example GKE with windows node pool (yes, I understand that it's not your use-case, but for further reference).

moonkotte
  • 3,661
  • 2
  • 10
  • 25
  • Thank you. I already saw the page "Adding Windows nodes" but found it hard to understand. Must the "Linux-based Kubernetes kubeadm cluster" a) be already spread over all my Windows hosts (=>how to create it ?!) or b) would it be sufficient to create it as a "single node cluster" on a single windows host? How do I do the "much more complicated networking"? – Stefan Mar 03 '22 at 10:54
  • Can the Docker-Kubernetes option be used as that "kubeadm-cluster"? – Stefan Mar 03 '22 at 11:02
  • 1
    Okay, there are two types of nodes in kubernetes - control-plane (which is a linux based machine where all kubernetes components are run) and worker nodes where only kubelet is presented which communicates with control plane. So `control-plane` shoudn't be spread on windows machines. Single `control-plane` for testing purposes is sufficient, for production it's recommended to have at least 3 control-plane (`etcd` is used as a data-storage and it requires having 3-5 nodes - you can read about [etcd](https://learnk8s.io/etcd-kubernetes) in k8s). – moonkotte Mar 03 '22 at 11:23
  • @Stefan here you can read about [high availability (HA) cluster](https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/). Docker desktop with kubernetes inits a single node cluster for testing/local development. It's not supposed to be added to the multi-node cluster. `kubeadm` is a different story, it's a tool for managing an on-premise cluster. – moonkotte Mar 03 '22 at 11:25
  • And last about "complicated networking" - I'm not a network engineer to give exact solutions how kuberneres control-plane inside a VM on one of the windows hosts will be communicating with other windows nodes. I depends on an existing network + routing from my understanding. Control plane on a separate VM exposes ports (which are presented on the link in my answer) and other nodes can communicate directly with it. Hope it clarifies your doubts – moonkotte Mar 03 '22 at 11:28
  • @Stefan A bit later reply, but you need: 1 - control plane installed (separate server or VM) + CNI (like flannel - in the link). 2 - prepare your windows nodes (install kubeadm + network patch, see links in the answer). 3 - adjust firewall accordingly (see links in the answer with ports which for control-plane and worker node). 4 - join the windows node to the cluster using the `kubeadm join` command which will be printed after `kubeadm init` was executed on the control-plane (which means cluster was created). 5 - fix/resolve networking stuff. – moonkotte Mar 03 '22 at 15:06
  • I tried to install kubeadm in a Docker container, following https://github.com/kubernetes/kubeadm/issues/17#issuecomment-277540628 However, I get an error [kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: ... connection refused. [kubelet-check] It seems like the kubelet isn't running or healthy. – Stefan Mar 07 '22 at 14:23
  • Furhtermore, the command ".\PrepareNode.ps1 -KubernetesVersion v1.23.0" gives me DownloadFile : Download https://github.com/rancher/wins/releases/download/v0.0.4/wins.exe failed At C:\WINDOWS\system32\PrepareNode.ps1:70 char:1 + DownloadFile "$global:KubernetesPath\wins.exe" https://github.com/ran ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,DownloadFile – Stefan Mar 07 '22 at 14:33
  • Well, you need to troubleshoot everything and ideally go to the appropriate github repos with exact issues. Like solution with running `kubeadm` in a container which is docker-in-docker is already non standard based on the k8s documentation. Why not to use a proper VM? At least to start with. Same for the second error - did you run it with administrator role? Again, you need to perform at least basic troubleshooting steps. – moonkotte Mar 08 '22 at 09:04