2

GNU parallel has some options to limit the memory usage, preventing memory leakage. Like --memsuspend and --memfree but it seems that the tool has not told how it detects the total amount of available memory?

I wonder if I enter a working environment assigned by a Job Scheduler like PBS pro with only 100GB working memory and the total memory available on that computation node is 500GB. So when my running jobs used 80GB, how does GNU parallel tell the available amount of RAMs? Is it 20GB or 420 GB?

Much appreciated

I tried to find relevant answers from the GNU parallel documentation but failed.

Pls provide relevant info if convenient.

Thanks!

Xscrapert
  • 21
  • 1

1 Answers1

1

It depends on your OS.

https://git.savannah.gnu.org/cgit/parallel.git/tree/src/parallel?h=20221022

Search for memfreescript.

On GNU/Linux it uses /proc/meminfo.

It is unaware of PBS limitations. If you find a way to make it aware (e.g. maybe there is an enviroment variable?) let me know.

(Also: it does not prevent memory leakage. Look at ulimit for that).

Ole Tange
  • 31,768
  • 5
  • 86
  • 104