Shards in Kubernetes

Question

We have an application that we are running in kubernetes. However, this application only support a fixed number of people.

It has multiple systems:

1. DB
2. Authentication system
3. Web Application
4. Media manager

We can scale the Web Application and the Media Manager on demand. However, the Authentication system is not designed as to scale up to higher numbers. This is a problem too because is a close system so is not possible to modify it.

However, the system can scale by itself, behaving like a shard. And we can send different users to each shard without problems.

How can we scale up these shards themselves?

The system now behaves like this:

1. 1 DB/Authentication deployment
2. 1 Web Application deployment
   1. HPA that checks the load of the app and scale it up as needed
3. Stateful set for Media manager
   1. HPA that scales it up as needed

So what we need is a higher level thing that will do:

1. Operator? Controller? Analyzing metrics maybe? That will scale up the following things:
   1. 1 DB/Authentication deployment
   2. 1 Web Application deployment
      1. HPA that checks the load of the app and scale it up as needed
   3. Stateful set for Media manager
      1. HPA that scales it up as needed

So as the load increases we are able to scale up these "shards" on demand.

I think this can be done with a controller or operator. But, is there any operator out there that already works like this?

Thanks for your help!

Answer 1

lets start with the deployments, since you are using a deployment i assume that these can be scaled easily
you can monitor these deployment by adding a sidecar container to their pods exposing metrics data then collecting the data and storing it in something like prometheus then you can use the prometheus to call a webhook whenever these metrics show latency on your database or high response time of any kind on your applications.
the application receiving the webhook will scale up or down your deployment via the kubernetes api (this application you should develop but it is fairly straight forward)

for the authentication system you can use the same strategy but a bit differently.
lets say you have 4 shards, create a separate deployment for each shard each having one shard and since it is sharded i assume a routing mechanism is in place.
configure the routing mechanism to distribute requests between these 4 deployments via their respective services.
collect metrics from authentication manager like other systems but when your system detects performance degradation reshard the authentication manager which the method for depends it it's behavior.
for example a mongodb sharded cluster will scale when you add more replica sets to a sharded cluster but a redis cluster needs shard rebalancing and hash slot modification when more performance is needed.
either way you need to create more deployments on demand which again is the job of the application receiving the webhook from prometheus.
also reconfigure the routing mechanism to connect to newly created shards as well.