0

Would like to clear about the pods resource consumption when its getting created or restarted as part of rolling update or scaling up.

looking to understand..

  • whether pods will consume entire resoources specified in its requests while its getting created? or limits ?
  • or it will just consume how much its required to start, which will be less than its request.

We are currently facing some issue with our AKS cluster that, pods generating high cpu usage alerts (morethan 95%) when new pods getting created as part of rollout or as part of scaling up , but our applications are light weight and needs less cpu for its functionality.

So looking for a solution for this ,

  • whether we can consider CPU initialization period /initial readiness, which will make the pods to manage its resource consumption during startups?

  • whether we can tweak the hpa settings during scaling up activities or any policy window or stabilization window during the pod startups?

Vowneee
  • 956
  • 10
  • 33

1 Answers1

2

That really depends on the resource type and the actual values you specify.

As your question focuses on CPU I will as well.

The first consideration is what kind of class your pod ends up in https://kubernetes.io/docs/tasks/configure-pod-container/quality-service-pod/

Pods generally do not "consume" any cpu resources, software running within them does, so what happens with CPU relies strictly on what software you're running. Some will have cpu heavy startup phase (oh what affection do I have for Java in k8s) and in that case initial spike of cpu will be perfectly normal, but also, due to scaling logic, that initial spike, if happening before pod is in ready state, will be discarded for computation of HPA scaleup.

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/

So my ultimate advice would be to set your readinessProbe correctly.

Radek 'Goblin' Pieczonka
  • 21,554
  • 7
  • 52
  • 48