1

I'm new to MPI and I want to ask how memory is distributed to all processes in a single computer.For example for a 4GB RAM and 2 processes, each process will take 2 GB of memory?

spyros
  • 127
  • 5

1 Answers1

0

There is no such a limit per process. It is up to the developer to let a process use less or memory than the other processes for example by branching based on the process rank.

if(rank == 0){
   vector<int> a(N);
   // do sth with a
}

now only a single process consumed some memory and the others did not.

Just bear in mind whatever you do all the processes will do if you don't branch per process. for example if you run the following piece of code with M processes:

vector<int> a(N);

you will consume M times more memory than a single process program. If the task to do some operation on a then you can use this instead:

vector<int> a(ceil((float)N/M));
Oblivion
  • 7,176
  • 2
  • 14
  • 33
  • I see, but the total memory in N processes must be less than RAM or it doesn't matter?I mean I can't have one process using 3GB of RAM and another one using 2GB(total RAM 4GB) or it doesn't matter? – spyros Jun 30 '19 at 20:23
  • @spyros look at what I did in the if statement as an example. A process can use memory more than the others. Vector a is scoped to rank 0 only. But if you don't use if everyone will allocate the same amount. – Oblivion Jun 30 '19 at 20:26
  • @oblivion your answer is great but I think spyros needs a book operating systems or an answer to help lead them there. – Bayleef Jun 30 '19 at 20:27
  • @oblivion I understand what you did and I got your point.My questions was: let's say we write a code in c++ using MPI and create 2 processes in a single computer can the "sum" of memory used by the 2 processes be bigger than total RAM – spyros Jun 30 '19 at 20:31
  • @spyros as bailey pointed out you may need a good book: https://stackoverflow.com/q/2275184/10933809 – Oblivion Jun 30 '19 at 20:31
  • @spyros You it is ok to try to allocate more memory than what you actually have just like a normal C++ program but the program most likely will crash. So yes you should not in total for all the processes use more memory than what you actually have. You also should take into account how many machines you have and how much memory each one has – Oblivion Jun 30 '19 at 20:37
  • @spyros no problem. Happy coding :). Just one more thing the processes running on a machine should not in total use more memory than what that machine has. – Oblivion Jun 30 '19 at 20:41