-1

I am trying to run a job using slurm, but the srun is only launching 1 process, the job script I am using is below

#!/bin/bash
#SBATCH -N 1
#SBATCH -p RM
#SBATCH -t 00:01:00
#SBATCH --ntasks-per-node=128
#SBATCH --error=job.err 
#SBATCH --output=job.out


srun -n 16 ./firstmpi > out

However, the output has only one process

Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors
Hello world from processor r309.ib.bridges2.psc.edu, rank 0 out of 1 processors

Please let me know how to solve the issue

Sap BH
  • 71
  • 1
  • 6
  • 1
    `srun` does launch the 16 processes you requested, but they run as 16 independent single task MPI jobs, instead of a single 16 tasks MPI job. That typically occurs if your MPI library cannot communicate with SLURM. For example, you would need to pass the path to `libpmi.so` to Intel MPI, or configure Open MPI with `--with-pmi` support. – Gilles Gouaillardet Aug 24 '23 at 03:09
  • I am using Mvapich2. – Sap BH Aug 24 '23 at 03:22
  • 1
    check https://mvapich.cse.ohio-state.edu/static/media/mvapich/mvapich2-2.3.6-userguide.html#x1-110004.3.3 for how to build mvapich with PMI support. `srun --mpi=list` will give you the list of what is supported by SLURM, if the default is not the good one, you might have to force it (e.g. `srun --mpi=pmi2 ./firstmpi`) – Gilles Gouaillardet Aug 24 '23 at 03:25

0 Answers0