Setting PBS/Torque/qsub parameters in script via command line arguments

Question

I want to be able to easily change how many nodes, ppn, etc I submit to qsub via script. That is, I want run somthing like this:

qsub script.sh --name=test_job --nodes=2 --ppn=2 --arg1=2

With a script like the following:

#/bin/bash
#PBS -N ${NAME}
#PBS -l nodes=${NODES}:ppn=${PPN},walltime=${WALLTIME}
#PBS -q ${QUEUE}
#PBS -m ${MAILOPTS}
#PBS -M ${EMAIL}

/some/command ${ARG1}

So, I want to be able to pass in arguments that both change the PBS environment as well as some that go to the executable itself.

I've tried using the -v argument of qsub:

qsub script.sh -v NAME=test_job,NODES=16,PPN=16,ARG1=2

But the job submitted with the name script.sh and 1 node, 1 ppn.

Any ideas on a solution to this?

score 5 · Accepted Answer · answered Feb 17 '17 at 18:06

@dbeer's answer gave me a little more insight. The solution to my problem is as follows:

#PBS args are overwritten by the command line. In such case, the args to PBS and the script itself must be separated. Therefore, rather than trying to do something like:

#PBS -l nodes=${NODES}:ppn=${PPN},walltime=${WALLTIME}

/some/command ${ARG1}

inside the script and running like

qsub script.sh -v NODES=2,PPN=2,WALLTIME=160:00:00,ARG1=2

these can all be set with args to qsub itself:

qsub script.sh -l nodes=2:ppn=2,walltime=160:00:00

Then, any args that need to be passed to the executable can be passed through the -v argument to qsub:

qsub script.sh -l nodes=2:ppn=2,walltime=160:00:00 -v ARGS1=2

score 4 · Answer 2 · answered Feb 17 '17 at 16:53

In Torque, all of the #PBS arguments are overridden when a matching argument is specified on the command line. For example, if your script has:

#PBS -l nodes=2

You can submit:

qsub script.sh -l nodes=4

and the command line will take precendence over the script. The docs have a complete list of the command line arguments.

Setting PBS/Torque/qsub parameters in script via command line arguments

2 Answers2