0

Following the tutorial on this link, I try to run a MPI code under MacOS 10.9.5 (Mavericks) with a number of process equal to 256 : the MPI code allocates for each process a 512x512 2D array, so it requires 256*256kB = 64MB of total used memory.

My MacOS has 16GB RAM and 8 cores, so it seems to be weird.

For a number of process lower than 256 (I tried : np=2,4,8,16,32,64,128), there is no problem, execution is good and I get expected results.

But for np = 256, I get the following message which repeats itself:

$ mpirun -np 256 ./a.out

    [warn] select: Invalid argument
    [warn] select: Invalid argument
    [warn] select: Invalid argument
     ...

I tried also to use -mca option by doing:

$ mpirun -mca opal_set_max_sys_limits 1 -np 256 ./a.out

But I get the same warning message.

From this link cited above, I did:

$ launchctl limit maxfiles
  
  maxfiles    65536          200000

Then, in root user, I created /etc/launchd.conf file and put into:

limit maxfiles 65536 200000

I restarted the system for the new limits to take effect and type as normal user:

$ launchctl limit maxfiles

maxfiles    65536          200000

But unfortunately, these modifications have no effects on the MPI function "mpirun" with 256 processes and don't make disappear the warning above.

On Linux platform, I can launch my MPI code with np = 256, without problem, the issue is only happening on MacOS 10.9.5.

Update 1

I have also tried to use "--oversubscribe" option like:

mpirun -np 256 --oversubscribe ./a.out

but get still the same message : [warn] select: Invalid argument

Update 2

I have upgraded to OS X 10.13.3 High Sierra but still have no luck with my issue.

To increase the limit of open files, from this link I did:

  1. sudo vim /Library/LaunchDaemons/limit.maxfiles.plist

  2. put into this file:

    Label limit.maxfiles ProgramArguments launchctl limit maxfiles 64000 524288 RunAtLoad ServiceIPC

  3. sudo launchctl load -w /Library/LaunchDaemons/limit.maxfiles.plist

  4. Checking by:

    $ sysctl kern.maxfiles

    kern.maxfiles: 524288

and:

$ launchctl limit maxfiles

maxfiles    64000          524288

When I launch ring_c.c (example from Open-MPI) executable with:

mpirun --oversubscribe -np $NB_PROCESS ./a.out

For a number of processes lower or equal to 142, it's running fine.

For a number of processes greater than 142, I get the original warning message:

[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
[warn] select: Invalid argument
...

Why the limit is reached at NB_PROCESS > 142? what does it correspond to? I didn't see this limit anywhere.

halfer
  • 19,824
  • 17
  • 99
  • 186
  • Please show a Minimal, Complete, Verifiable version of your code. – Mark Setchell Feb 14 '18 at 19:50
  • Try `ulimit -S -n 49000` and `ulimit -S -u 2048` before running. – Mark Setchell Feb 20 '18 at 01:56
  • -@Mark Setchell : I tried your commands but issue remains for a number of processes > `142` (NB_PROCESS) with : `mpirun --oversubscribe -np $NB_PROCESS ./a.out`. You can test the MPI code donwloadable on : http://31.207.34.24/test_mpi/ring_c.c –  Feb 20 '18 at 03:11
  • Works fine with open-mpi 3.0.0 on High Sierra including 512 processes. `brew install open-mpi` and `ulimit` per my previous comment. – Mark Setchell Feb 20 '18 at 08:02
  • If you have upgraded macOS, I would suggest you uninstall `open-mpi`, update `Xcode` if necessary, re-run `xcode-select --install`, re-install `open-mpi` and re-compile. – Mark Setchell Feb 20 '18 at 09:10
  • I am using `macport` but I think this should work like with `brew`. Which compiler have you used for your `open-mpi` installation ? –  Feb 20 '18 at 09:54
  • I used `gcc 7.2` – Mark Setchell Feb 20 '18 at 09:58
  • I built `open-mpi 3.0.0` with `gcc-mp-7` (7.3) and set :`ulimit -a` `core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited file size (blocks, -f) unlimited max locked memory (kbytes, -l) unlimited max memory size (kbytes, -m) unlimited open files (-n) 49000 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 2048 virtual memory (kbytes, -v) unlimited` **unfortunately,still message** –  Feb 20 '18 at 10:49
  • Finally fixed by using external libevent for open-mpi-3.0 (built from libevent-2.1.8 version) and the following configure : `./configure --prefix=/usr/local/openmpi CC=gcc-mp-7 CXX=g++-mp-7 FC=gfortran-mp-7 --with-libevent=external --with-libevent=/usr/local/libevent/` –  Feb 20 '18 at 12:43

0 Answers0