1

I have a project where I have multiple nodes in a grid that are operating as storage and compute nodes. These nodes are heterogeneous ranging from dedicated server hardware to alix boards and even Mikrotik routerboards running OpenWRT(consisting of mostly the alix boards though).

I am collecting system performance metrics such as RAM, SWAP and CPU usage through the sigar java API. This is working really well.

My next task is to take these performance values and combine them in a way that I would be able to sort my hosts in such a way that I could say which host is best able to handle a compute request. This can almost be thought of as the same way windows gives your computer a performance score with the windows performance index.

My first attempt at this was to add the used percentages of RAM, SWAP and CPU usage ( for Linux I scaled the load value to a value between 0 and 1). Whichever host scored the lowest was then selected for the compute operation.

Has anyone got a better idea on how to do this. My way felt very "hacky" and not the sort of way I want to approach the work in this project.

Thanks

tensai
  • 1,628
  • 3
  • 17
  • 22

1 Answers1

1

I would get an estimate of the requirements of task and place a machine to match. You might have many small jobs and one big job. If you allocate the small jobs first, the biggest server might be busy while the smaller boxes might be too small. You might want to allocate the small tasks to the small servers and leave the big machines for the bigger tasks.

You will also have tasks which require more CPU or more memory.

I wouldn't consider SWAP usage. Java cannot run in SWAP space so you need to ensure it never needs it.

Peter Lawrey
  • 525,659
  • 79
  • 751
  • 1,130
  • My supervisor wants me to use a greedy approximation algorithm (used for solving the knapsack problem) in order to allocate tasks to specific nodes. Also the nodes are not bound to only consume one specific task, the applications are multithreaded and can handle more than one request to the node. – tensai Jan 15 '13 at 18:50