0

This a VERY open question.

Basically, I have a computing application that launches test combinations for N Scenarios.
Each test is conducted in a single dedicated thread, and involves reading large binary data, processing it, and dropping results to DB.

If the number of threads is too large, the app gets rogue and eats out all available memory and hangs out.. What is the most efficient way to exploit all CPU+RAM capabilities (High Performance computing i.e 12Cores/16GB RAM) without putting the system down to its knees (which happens if "too many" simultaneous threads are launched, "too many" being a relative notion of course)

I have to specify that I have a workers buffer queue with N workers, every time one finishes and dies a new one is launched via a Queue. This works pretty fine as of now. But I would like to avoid "manually" and "empirically" setting the number of simultaneous threads and have an intelligent scalable system that drops as many threads at a time that the system can properly handle, and stop at a "reasonable" memory usage (the target server is dedicated to the app so there is no problem regarding other applications except the system)

PS : I know that .Net 3.5 comes with Thread Pools and .Net 4 has interesting TPL capabilites, that I am still considering right now (I never went very deep into this so far).

PS 2 : After reading this post I was a bit puzzled by the "don't do this" answers. Though I think such request is fair for a memory-demanding computing program.

EDIT
After reading this post I will to try to use WMI features

Community
  • 1
  • 1
Mehdi LAMRANI
  • 11,289
  • 14
  • 88
  • 130
  • 2
    The thread pool has been in .NET since 1.0 – Richard Blewett Jan 28 '12 at 16:35
  • Sounds simple to me if maximizing perf is the only goal. You never want to have more threads than you have cpu cores. If it turns out that you don't have enough RAM then the hardware is imbalanced. Add more RAM. Don't forget the reality check, actually check that all cores are maxed out at 100%. If they are not then threads are not the solution. And using too many threads just creates problems. – Hans Passant Jan 28 '12 at 16:51
  • @HansPassant that's true, but only if the threads don't block (file i/o, etc..), if so more may help keep the performance up. – kenny Jan 28 '12 at 16:56
  • 2
    @kenny - If there are blocking threads then the bottleneck is elsewhere. Invariably I/O. Adding more threads doesn't make I/O faster. – Hans Passant Jan 28 '12 at 17:03
  • @HansPassant true, but it can make the CPU do other operations that aren't blocked. – kenny Jan 28 '12 at 20:19

1 Answers1

2

All built-in threading capabilities in .NET do not support adjusting according to memory usage. You need to build this yourself.

You can either predict memory usage or react to low memory conditions. Alternatives:

  1. Look at the amount of free memory on the system before launching a new task. If it is below 500mb, wait until enough has been freed.
  2. Launch tasks as they come and throttle as soon as some of them start to fail because of OOM. Restart them later. This alternative sucks big time because your process will do garbage collections like crazy to avoid the OOMs.

I recommend (1).

You can either look at free system memory or your own processes memory usage. In order to get the memory usage I recommend looking at private bytes using the Process class.

If you set aside 1GB of buffer on your 16GB system you run at 94% efficiency and are pretty safe.

usr
  • 168,620
  • 35
  • 240
  • 369
  • Yeah 2 Definitely Sucks :-) And 1 is what I was about to do. But I wanted to see how many stones I would get thrown at while being called a heretic before doing it ^^ Any code available ? – Mehdi LAMRANI Jan 28 '12 at 16:37
  • Just stumbled upon WMI capabilities spotted on this post : http://stackoverflow.com/questions/3296211/c-how-to-get-the-size-of-available-system-memory reference here : http://msdn.microsoft.com/en-us/library/windows/desktop/aa394181%28v=vs.85%29.aspx sounds interesting. – Mehdi LAMRANI Jan 28 '12 at 16:43
  • I have added a remark on how you can find out about memory usage. You will do fine with a manually guessed memory limit. Just be sure to send yourself an email whenever you get an OOM so you can adjust. – usr Jan 28 '12 at 16:44
  • Oh ok I see. I could use the WorkingSet64 and limit it to a percent of the total system memory. I have to use VisualBasic.Devices.ComputerInfo.TotalPhysicalMemory (which is not VB specific although it sounds confusing) as suggested by Hans Passant in the pos i quoted in the edit – Mehdi LAMRANI Jan 28 '12 at 16:55
  • 1
    No, don't use the working set. The working set is the amount of memory currently paged in (roughly speaking). It can change without your app doing anything! Be very careful with memory counters... Use private bytes as I said. Private bytes is that memory that is not shared with other processes. It roughly corresponds to your allocations. Unfortunately, neither on Windows nor on Linux there is a single counter that can tell you the real memory usage (such a thing cannot exist due to sharing). – usr Jan 28 '12 at 16:58