2

I would like to know how to submit several mpirun programs in one job script file in Sun Grid Engine.

Suppose I have compiled program1, program2, program3 .. program100 and want to run them simultaneously such that each use 16 cpus in one job script without making 100 job scripts.

mpirun -np 16 program1 &
mpirun -np 16 program2 &
mpirun -np 16 program3 ....

How can I distribute nodes to each programs in Sun Grid engine? (Can I split hostfile of SGE into 100 and supply them into each mpirun? )

Thank you.

Konrad Krakowiak
  • 12,285
  • 11
  • 58
  • 45
user2775514
  • 191
  • 1
  • 2
  • 7

1 Answers1

1

The proper way to implement that use case with SGE is to use array jobs:

#$ -cwd
#$ -pe mpi 16
#$ -t 100

mpirun -np $NSLOTS ./program${SGE_TASK_ID}

The -t 100 parameter creates an array job of 100 tasks and each one receives a different value in the SGE_TASK_ID environment variable ranging from 1 to 100.

Splitting the hostfile is undesirable since that could interfere with the integration between the MPI library and the SGE execution and job monitoring mechanisms.

Hristo Iliev
  • 72,659
  • 12
  • 135
  • 186
  • I tried the array job in SGE and it worked. Array job is automatically repeating the same job script many times with some name convention. Thank you. – user2775514 May 28 '15 at 06:26