1

I am working on designing a solution with HA and load distribution. I have a question about how the relationship can be built between a worker node and master node to support HA (Fault Tolerance) and load distribution.

My understanding that a worker node is managed by a master node which means that during the building of a worker node, we make it join a master node in order to build the trust and link with the master node.

So now if I want to build HA for the cluster on the master node:

  • I am going to define 3 master nodes, each has an API server with a loadbalancer in-front of them.
  • build ETCD cluster externally having 3 nodes and make the master nodes uses them.
  • the 3 master nodes will have each a scheduler.

Going to the worker node:

  • I start the worker node by making it join to a master using the VIP/loadbalanced API-server.

So my questions are:

  • on the 3 master nodes, should we have a controller?
  • Can we link the worker node to the 3 master nodes? if Yes, What is the benefit of that? Isn't it better to use the loadbalancer/VIP? Will, in that case, we still have the limitation of 5000 nodes? or can it be up to 5000X3 worker nodes?
  • Can we have a cluster of API-server with active-passive just like we do in etcd? even if we can do it just using a set of configurations in the initialization phase?
Rico
  • 58,485
  • 12
  • 111
  • 141
rasadus
  • 31
  • 2

2 Answers2

0

on the 3 master nodes, should we have a controller?

I'm not sure what controller you are talking about (A Kubernetes controller? The kube-controller-manager?). In summary, all the master nodes would have the same Kubernetes components: kube-apiserver, kube-scheduler, kube-controller-manager, kube-proxy, runtime, etc.

Can we link the worker node to the 3 master nodes? if Yes, What is the benefit of that? Isn't it better to use the loadbalancer/VIP? Will, in that case, we still have the limitation of 5000 nodes? or can it be up to 5000X3 worker nodes?

If you have a 3 master setup, one is active (leader) and the others are in standby, there's only one leader established by a configured quorum for these components, so it's not that you can scale 5000x3 nodes, the limit of 5000 would still apply because you are always talking to the leader.

Can we have a cluster of API-server with active-passive just like we do in etcd? even if we can do it just using a set of configurations in the initialization phase?

That's the way it works. In a 3 master setup, there's 1 active and 2 passive, in a 5 master set up there 1 active and 5 passive.

Hope it helps!

Rico
  • 58,485
  • 12
  • 111
  • 141
0

Yes I am talking about the kube control manager. Now I know that it will run in a leader slave mode, I mean the kube control manager as well the kube schedule. so only one master is leading those.

Yes I was thinking as well that it is not going to scale above the 5000 (the limits of a master node).

I am not sure that kube api server runs in a leader slave cluster that is why I believe we need to put a solution above it to hide the real api server ip or dns.

I was more told by a colleague that we can have multiple api servers to the same worker node, I have not tried it myself but I truly suspect that it will work and I still believe the option of a DNS server just like Ruote 53 or a Proxy like nigix or even a hardware balancer as F5 should be the possible options to let the worker communicate with the active master without depending on it.

rasadus
  • 31
  • 2