Horizontal Pod autoscaler not scaling down

Question

I've got my HPA like this. The pods are scaling well but its not scaling down and the CPU utilization is just 5%. It is not scaling down to the minimum of 1 replica.

The below is my HPA

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: app
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: AverageValue
        averageValue: 100Mi

Is memory utilization of your Pods less than 90Mi? Also, there's is some delay in scaling. — Grigoriy Mikhalkin, Nov 21 '20 at 11:31
Memory - 205690880m/100Mi, and it's been like 15 minutes with no load — jeril, Nov 21 '20 at 11:32
So it's about 200Mi of memory per Pod. That's why it's not scaling down — Grigoriy Mikhalkin, Nov 21 '20 at 11:35
In [Kubernetes HPA doesn't scale down after decreasing the loads](https://stackoverflow.com/questions/64138144/kubernetes-hpa-doesnt-scale-down-after-decreasing-the-loads/64185360#64185360) I note that, if you specify multiple metrics, the largest replica count for any metric determines the scaling value. — David Maze, Nov 21 '20 at 11:37
@DavidMaze That's not true. [HPA scale down](https://github.com/kubernetes/kubernetes/issues/95861#issuecomment-729082546) when all metrics values are below certain threshhold. — Grigoriy Mikhalkin, Nov 21 '20 at 11:42
@DavidMaze But you can only set threshold value globally for all HPAs. If in your case your need to set this threshold to specific HPA, please, write about your case in [this issue](https://github.com/kubernetes/kubernetes/issues/95861) — Grigoriy Mikhalkin, Nov 21 '20 at 11:45
Many applications also do not so quickly release memory. I would check that yours does before scaling by memory use — Justin Tamblyn, Nov 22 '20 at 09:06
Have you checked if the scaling is behaving as you want if you delete the average memory value? How many replicas this `Deployment` is scaling too in this setup? — Dawid Kruk, Nov 23 '20 at 17:50
The problem was scaling down if there is no load. yes it works perfectly if I remove the memory values..The pods were not releasing memory.. I just created 10 replicas — jeril, Nov 24 '20 at 04:44
@jeril I assume that by your last comment the issue you were facing was discovered (memory not being released by app). Could you please provide the findings in an answer so that it could be more visible for the community? — Dawid Kruk, Dec 01 '20 at 16:44

score 1 · Accepted Answer · answered Dec 02 '20 at 05:09

In the above yaml file, I was using two resources as shown below

metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: AverageValue
        averageValue: 100Mi

The memory resource was not quickly released by the application and once I removed the memory resource the pods scaled down when the utilization was less.

Horizontal Pod autoscaler not scaling down

1 Answers1