Kubernetes scaling based on network utilization or requests per second

Question

Is there any way to scale Kubernetes nodes based on network utilization and not based on memory or CPU?

Let's say for example you are sending thousands of requests to a couple of nodes behind a load balancer. The CPU is not struggling or the memory, but because there are thousands of requests per second you would need additional nodes to serve this. How can you do this in Google Cloud Kubernetes?

I have been researching around but I can't seem to find any references to this type of scaling, and I am guessing I am not the only one to come across this problem. So I am wondering if any of you knows of any best practice solutions.

I guess the ideal solution would be to have one pod per node receiving requests and creating more nodes based on more requests and scale up or down based on this.

Run kubectl scale based on the network traffic metrics without relying on K8S? — mon, Feb 10 '18 at 09:10
The Horizontal Pod Autoscaler supports [custom metrics](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-custom-metrics). I cannot give a more detailed answer, since I never worked with it. — svenwltr, Feb 10 '18 at 10:29
Cluster autoscaler is based on [POD allocation](https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler).I believe @svenwltr is pointing to the right direction. According to [this discussion](https://stackoverflow.com/questions/37080797/google-container-engine-how-to-auto-scale-an-instance-group-based-on-http-load/37082052#37082052) autoscaling based on the HTTP load is not supported. Horizontal Pod autoscaler and [custom metrics](https://groups.google.com/forum/#!topic/kubernetes-users/3hHvmK_5AjE) seems to be the best approach. — Carlos, Feb 11 '18 at 23:20

score 5 · Answer 1 · answered Aug 04 '20 at 11:34

5

This is possible and you have to use Prometheus Adaptor to configure custom rules to generate Custom Metrics.

This link has more details on how to setup prometheus, install adaptor and apply configuration with custom metrics..

answered Aug 04 '20 at 11:34

Rohit

1,231
10
22

score 2 · Answer 2 · answered Jan 31 '21 at 04:11

I've implement this on my gke cluster using this custom metrics.

This the example of my HPA configuration :

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
  name: hpa-name
  namespace: your-namespace
  annotations:
    metric-config.external.prometheus-query.prometheus/interval: 30s
    metric-config.external.prometheus-query.prometheus/prometheus-server: http://your-prometheus-server-ip
    metric-config.external.prometheus-query.prometheus/istio-requests-total: |
      sum(rate(istio_requests_total{reporter="destination", destination_workload="deployment-name", destination_service_namespace="your-namespace"}[2m]))
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: deployment-name
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: External
    external:
      metric:
        name: prometheus-query
        selector:
          matchLabels:
            query-name: istio-requests-total
      target:
        type: AverageValue
        averageValue: 7

score -1 · Answer 3 · answered Oct 06 '20 at 17:22

-1

I think HPA(Horizontal Pod Autoscaler) along with Cluster Autoscaler will do the magic.

Have a look at this - https://medium.com/google-cloud/kubernetes-autoscaling-with-istio-metrics-76442253a45a

answered Oct 06 '20 at 17:22

Himadri Ganguly

715
4
11
31

Kubernetes scaling based on network utilization or requests per second

3 Answers3