2

I am trying to write a PBS script.

I have to use 4 nodes in a cluster and to run 4 times an executable program. The difficulty is that the 4 executions have to run in parallel, one by node

For the moment, I have just allocated the nodes

#PBS -l nodes=4:ppn=1

The problem is I don't know how to be sure the programs run in parallel. How can I do it ?

federem
  • 299
  • 1
  • 6
  • 17
  • Are you writing an MPI program? Does your program communicate somehow? Launching your program with PBS will essentially make 4 copies of your program on different nodes of your cluster, but it won't make them communicate. You need to use something like MPI that allows them to execute together. – Wesley Bland Oct 07 '13 at 16:15
  • Moving this to an answer at it's more clear what you're asking now and I can't fit my response in the character limit. – Wesley Bland Oct 07 '13 at 17:27
  • Does this thread help? http://stackoverflow.com/questions/18948727/naive-parallelization-in-a-pbs-file – dbeer Oct 07 '13 at 20:48

2 Answers2

1

If you don't need your program to communicate with other copies of itself, you don't need MPI. MPI stands for the Message Passing Interface (MPI Forum Website) and is the most popular way of writing a parallel program. There are lots of tutorials on the web that you can find with a quick search. I recommend taking a look at them. You can test things out on your own machines by installing an MPI library yourself. They're available in most package managers. The most popular open-source implementations are MPICH and Open MPI. If you're starting from scratch, try to get the most up to date versions as you'll get more help from the various communities that way.

You can execute a non-MPI program with mpiexec and it will launch across multiple nodes (assuming you've set things up correctly), but it won't do anything to make them communicate. That has to be done explicitly (again, see the MPI tutorials).

If you're not going to communicating between the processes (parallel computing), I have to wonder why you want to run multiple copies of the same program across the cluster to begin with. The only thing that comes to mind are simple cluster maintenance utilities, but there are much better ways of doing that than PBS/MPI setups and I'd imagine that if you were doing cluster maintenance, you'd already know what to use for that.

Wesley Bland
  • 8,816
  • 3
  • 44
  • 59
1

pbsdsh is a very simple way to accomplish what you're talking about. 

# this will launch one copy of <executable> on each execution slot. For -l nodes=4:ppn=1
# it will launch it once per node.
pbsdsh <executable>

Sometimes you have to supply the full path to pbsdsh depending on your environment. It defaults to /usr/local/bin/pbsdsh.

dbeer
  • 6,963
  • 3
  • 31
  • 47