1

Please help in solving the following problem.

The following entities are given:

  1. Application. Applications reside on storage and they generate traffic through service node.
  2. Service. Service is divided into several nodes. Each node has access to local or/and shared storage.
  3. Storage. This is where applications resides. It can be local (connected to only one service node) or shared by several nodes.

Rules:

  1. Each application is placed on some particular storage. And the storage cannot be changed.
  2. The service node for the application can be changed to another one as long as new service node has access to the application's storage.

For example, if App resides on local storage of Node0, it can only be served by Node0. But if App resides on storage shared0, it can be served by Node0, Node1 or Node2.

The problem is to find the algorithm to rebalance applications between service nodes, given that all applications are already placed on their datastores. And to have this rebalancing as fair as possible.

If we take for example shared2 storage, the solution seems trivial: we take the apps count for Node3 and Node4 and divide all apps equally between them. But when it comes to shared1 it becomes more complicated since Node2 also has access to shared0 storage. So when rebalancing apps from group [Node2, Node5] we also have to take into account apps from group [Node0, Node1, Node2]. Groups [Node2, Node5] and [Node0, Node1, Node2] are intersecting and rebalancing should be performed for all groups at once.

I suspect that there should be well-known working algorithm to this problem, but still cannot find it. enter image description here

alexlys
  • 49
  • 5

1 Answers1

1

I think the Hungarian Matching algorithm would fit your needs. However, it might be a simple enough problem to try your own approach.

If you separated all the unconnected graphs, you'll have some set of Shared storage units per graph, each set being associated with a collection of Apps. If you spread each of those Apps evenly across each Storage's associated Nodes, you would have some Nodes with more Apps than others. Those nodes will be connected to multiple Shared storage units.

If all vacant nodes are filled, there should always be a transitive relationship between any two Nodes within a connected graph such that an App from one can be decreased ann an App from the other can be increased, even if some intermediate displacements are needed. So, if you iteratively move an App along the path from the heaviest Node to the lightest Node, shortcutting if you reach a vacant Node, and swapping Apps at any intermediate node as needed to continue along that path through one or more Shared storage units, you should be balanced when the count of the heaviest and lightest nodes differ by no more than one.

phatfingers
  • 9,770
  • 3
  • 30
  • 44
  • Thank you for the interest in my question! Hungarian method is about the total cost but the need is to balance a cost per entity. The problem can be rephrased in the following way - "You have a set of balls and a set of boxes and you have the restriction that not every ball can be put in every box. Try to minimize the difference of the number of balls among the boxes." – alexlys Feb 10 '22 at 10:00
  • At the moment what I could find is this paper that seems to be close to the problem https://qspace.qu.edu.qa/bitstream/handle/10576/8020/06-02-15-0012-fulltext.pdf?sequence=8 – alexlys Feb 10 '22 at 10:03