1

I am using minikube on Windows, there is only one node "master".

The spring boot application deployed has REST endpoint which gives the number of client its currently serving. I would like to scale out horizontally or auto spin a pod when the requests reaches some limit.

Lets say:
There is 1 pod in the cluster.
If the request limit reached 50 (for Pod 1), spin up a new pod.
If the request limit reached 50 for Pod 1 and Pod 2, spin up a new Pod (Pod 3).

I tried researching on how to achieve this, I was not able to figure out any. All I could find was scaling out using CPU usage with HorizontalPodAutoscaler(HPA). Would be helpful to receive a guidance on how to achieve this using Kubernetes HPA.

Anthony
  • 205
  • 1
  • 2
  • 12
  • usually autoscaling the nodes comes from the underlying infrastructure, if using AWS then autoscaling groups if using Azure then Scaling Sets. If you want to scale PODs then you have to start looking at kubernetes ingress controllers, they can work with ingress load balancers to determine whether to spin up a new Pod – Callum Linington Sep 11 '19 at 08:28
  • 1
    @CallumLinington, horizontal pod autoscaler is exactly for scaling pods. ingress and node autoscaling has nothing to do with it. – Markus Dresch Sep 11 '19 at 08:48
  • Apart from HPA is only used around CPU utilisation - not for request limit which is what the OP is asking for. And seeing as Load Balancers are tracking connections they would be perfect for identifying the need to spin up new nodes – Callum Linington Sep 11 '19 at 09:12
  • The application already tracks the connections and there is a REST api when hit gives the count of connections. If required I can use the connection counts tracked by Load Balancer. Anyway is fine for me, my usecase is to scale out pods when the connection reaches a threshold limit. – Anthony Sep 11 '19 at 09:18

1 Answers1

1

I believe you can start from the autoscaling on custom metrics article. As per I see - this is achievable using the custom metrics in conjunction with Prometheus Adapter for Kubernetes Metrics APIs (An implementation of the custom.metrics.k8s.io API using Prometheus).

Prometheus Adapter for Kubernetes Metrics APIs repo contains an implementation of the Kubernetes resource metrics API and custom metrics API.

This adapter is therefore suitable for use with the autoscaling/v2 Horizontal Pod Autoscaler in Kubernetes 1.6+.

Info from autoscaling on custom metrics:

Notice that you can specify other resource metrics besides CPU. By default, the only other supported resource metric is memory. These resources do not change names from cluster to cluster, and should always be available, as long as the metrics.k8s.io API is available.

The first of these alternative metric types is pod metrics. These metrics describe pods, and are averaged together across pods and compared with a target value to determine the replica count. They work much like resource metrics, except that they only support a target type of AverageValue.

Pod metrics are specified using a metric block like this:

type: Pods
pods:
  metric:
    name: packets-per-second
  target:
    type: AverageValue
    averageValue: 1k

The second alternative metric type is object metrics. These metrics describe a different object in the same namespace, instead of describing pods. The metrics are not necessarily fetched from the object; they only describe it. Object metrics support target types of both Value and AverageValue. With Value, the target is compared directly to the returned metric from the API. With AverageValue, the value returned from the custom metrics API is divided by the number of pods before being compared to the target. The following example is the YAML representation of the requests-per-second metric.

type: Object
object:
  metric:
    name: requests-per-second
  describedObject:
    apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    name: main-route
  target:
    type: Value
    value: 2k

Also maybe below will be helpful for your future investigations:

Autoscaling on more specific metrics

Autoscaling on metrics not related to Kubernetes objects

Hope it helps

Vit
  • 7,740
  • 15
  • 40
  • Thank you for the inputs. Autoscaling is not a issue. But the major challenge is to route to the specific pod which accecpted the request. Please see this : https://stackoverflow.com/q/58007073/11836838 – Anthony Sep 30 '19 at 06:33
  • @VKR I am new to Kubernetes and confused with HPA. Do we need Prometheus and Prometheus adapter with autoscaling/v2beta2? I am using spring boot actuator and Prometheus at the moment and it is working fine to auto scale on number of request – Chandresh Mishra Nov 23 '19 at 16:29