2

UPDATE: I'm deploying on AWS cloud with the help of kops.

I'm in the process applying HPA for one of my kubernete deployment. While testing the sample app, I deployed with default namespace, I can see the metrics being exposed as below ( showing current utilisation is 0%)

$ kubectl  run busybox --image=busybox --port 8080           -- sh -c "while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; \
         env | grep HOSTNAME | sed 's/.*=//g'; } | nc -l -p  8080; done"


$ kubectl get hpa
NAME          REFERENCE                TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
busybox       Deployment/busybox       0%/20%    1         4         1          14m

But when I deploy with the custom namespace ( example: test), the current utilisation is showing unknown 

 $ kubectl get hpa --namespace test
NAME        REFERENCE            TARGETS                          MINPODS   MAXPODS   REPLICAS   AGE
busybox     Deployment/busybox   <unknown>/20%                    1         4         1          25m

Can someone please suggest whats wrong here?

Nagarjuna D N
  • 541
  • 6
  • 26

2 Answers2

1

For future you need to meet few conditions for HPA to work. You need to have metrics server or heapster running on your cluster. What is important is to set resources on namespace basis.

You did not provide in what environment is your cluster running, but in GKE by default you have a cpu resource set (100m), but you need to specify it on new namespaces:

Please note that if some of the pod’s containers do not have the relevant resource request set, CPU utilization for the pod will not be defined and the autoscaler will not take any action for that metric.

In your case I am not sure why it does work after redeploy, as there is not enough information. But for future remember to:

1) object that you want to scale and HPA should be in the same namespace

2) set resources on CPU per namespace or simply add --requests=cpu=value so the HPA will be able to scale based on that.

UPDATE:

for your particular case:

1) kubectl run busybox --image=busybox --port 8080 -n test --requests=cpu=200m -- sh -c "while true; do { echo -e 'HTTP/1.1 200 OK\r\n'; \ env | grep HOSTNAME | sed 's/.*=//g'; } | nc -l -p 8080; done"

2) kubectl autoscale deployment busybox --cpu-percent=50 --min=1 --max=10 -n test

aurelius
  • 3,433
  • 1
  • 13
  • 22
  • Updated my question. Thanks! – Nagarjuna D N Apr 01 '19 at 15:50
  • it is the same in kops, default ns has 100m CPU resource, and if you create a new one it has none until you set new. Still don't know why it works after redeploy. Are you sure it is correct namespace you are checking? I added correct steps to my answer for you. – aurelius Apr 01 '19 at 16:03
  • Yes, you are right. I followed both of the points what you suggested. While setting up for the first time it didn't work with the custom namespace. But it works when I redeploy everything including kops(kubernete) cluster. Might be silly mistakes. Anyways, I appreciate your response. Thanks! – Nagarjuna D N Apr 01 '19 at 16:07
0

Try running the commands below in the namespace where you experience this issue and see if you get any pointers.

  • kubectl get --raw /apis/metrics.k8s.io/ - This should display a valid JSON
  • Also, do a kubectl describe hpa name_of_hpa_deployment - This may indicate if there are any issues with your hpa deployment in that namespace.
Lateef Koshemani
  • 51
  • 6
  • Thanks for the response! Redeploying everything including kubernete cluster works. Still I'm not sure why it didn't for the first time – Nagarjuna D N Apr 01 '19 at 15:01