1

Is there a way to pause/resume kubernetes pods. We are deploying our pods in 2 regions (west and east). we want west pods to be on standby. Though we are able to restrict http traffic to just one region, the schedulers, cronjobs and backend processing runs in both the regions.

Any help would be appreciated.

user1717764
  • 111
  • 3
  • 12
  • 1
    You can do things like `kubectl scale deployment ... --replicas=0` to shut off a group of pods, and then increase the replica count again to recreate them when you need them. – David Maze Sep 24 '20 at 15:34
  • @david thanks for the reply. But to scale up the replicaset again, we will have the downtime. It will take 15-20mins for all the pods to scale up – user1717764 Sep 24 '20 at 16:13
  • @DavidMaze we are trying to failover without downtime. Scaling up replicas during failover will have a considerable downtime – user1717764 Sep 24 '20 at 16:46
  • I don't have an answer to your question but I have a suggestion you could start pods in both region and have only one region receiving traffic. You could switch DNS and forward traffic to another region for failover. This way the pods are already there and downtime would the time take for DNS to get updated depends on TTL – Rohit Sep 25 '20 at 06:22
  • I am not sure why are you trying to do this. You are still paying for allocated resources in the second region so why can't you use it? @user1717764 – Matt Sep 25 '20 at 08:18
  • @Matt we have infra challenges. We have database in just one region. Cross region access is slower. we don't want to use that all the time and we want it as a standby – user1717764 Sep 25 '20 at 13:10
  • @Rohit Thanks for your suggestion. We are actually restricting http traffic to one region already. We have crons and backend process which we are not able to restrict to one region – user1717764 Sep 25 '20 at 13:14
  • "We have database in just one region" - but if first region goes down, so does the database and how do you expect to operate in this case? @user1717764 – Matt Sep 28 '20 at 10:47
  • Anyway, no, its not possible to *pasue* pods without using some custom solution. You could for example write a controler that would watch the cluster and alter [service endpoins](https://kubernetes.io/docs/concepts/services-networking/service/#services-without-selectors) for the pods based on what is happening with given region. – Matt Sep 28 '20 at 10:59

1 Answers1

0

I have recently found out about k8s Service Topology and it looks like it might solve your problem. Unfortunately, this is an alpha feature since Kubernetes v1.17, and it may not work as expected.

Service Topology enables a service to route traffic based upon the Node topology of the cluster. For example, a service can specify that traffic be preferentially routed to endpoints that are on the same Node as the client, or in the same availability zone.

Matt
  • 7,419
  • 1
  • 11
  • 22