How to set number of threads as a downstream variable in PBS job queue

Question

Is there any way to determine how many threads are available to a program when I run it from a PBS job script?

In the header of my PBS job script I set

#PBS -l nodes=1:ppn=8

Is there a command I can use that will detect the number of threads - so that I can set a variable to equal this number (for downstream processes).

This way I can set threads as $k, for downstream processes, instead of going through the code line by line every time I change #PBS -l nodes=1:ppn=_.

Thanks all! I found a workaround - So if using a single node the variable I am looking for is $PBS_NUM_PPN

@jww - This makes more sense if you understand the context: PBS is Portable Batch System (or possibly Torque), and `#PBS` "comments" are how you specify scheduler parameters for your job. — Stephen C, Jan 05 '19 at 00:34
@Adi -- PBS is not part of bash itself, or a tool available on out-of-the-box Linux installs, and an overwhelming majority of bash (and Linux) users have no idea that it even exists. Thus, to get more clued-in answers (and fewer downvotes), it's helpful to emphasize the PBS part more than the bash or Linux parts when asking questions that relate to both; I've edited appropriately. — Charles Duffy, Jan 05 '19 at 01:09

Stephen C · Answer 1 · 2019-01-05T01:51:19.543

By default, PBS doesn't expose the ppn setting in the running job. And there is no way that a shell script can read its comments ... without knowing and parsing its source (and that's probably not going to work here for a couple of reasons.)

But here are a couple of ideas:

You could pass an arbitrary variable from the qsub command line using the -v option. (You might be able to do the same thing using #PBS -v ... but would be equivalent to setting a variable in your script in the normal way.)
You should be able to specify the resources (using -l) on the qsub command line instead of in the job script.

Put them together like this:

qsub ... -l nodes=1:ppn=8 - v NOSTHREADS=8 myscript.pbs

where myscript.pbs is:

#!/bin/bash
#PBS directives ... without the "-l" !!!

# ordinary shell commands.

somecommand --someoption $NOSTHREADS ...

Note: I recommend that you don't mix specifying resources on the command line and in the script. Put the "-l" options in one place only. If you put them in both places AND your Torque / PBS installation uses job submission filters, things can get rather confused.

Alternatively, you could write a shell (or python or whatever) launcher that generates the PBS script with matching values of the ppn (etc) resource(s) and the corresponding variable(s) embedded in the generated script.

This approach can have the advantage of being more reproducible ... if you do a few other things as well. (Ask a local eResearch analyst about reproducibility in your scientific computing.)

If neither of the above can be made to work, you might be able to check the ulimit settings within the job script. However, my understanding is that the PBS mon will typically not use ulimit restrictions as the means of enforcing thread / process limits. Instead, it will monitor the number of cores that are active. (The ppn resource limits the number of processors, not the number of threads or processes.)

I don't know PBS enough to speak to this with any certainty, and this probably falls within the "couple of reasons" you elided -- but at runtime, will `$BASH_SOURCE` refer back to a file with comments intact, or is PBS feeding the script to the shell via a FIFO or other mechanism that doesn't allow inspection of the linked file? — Charles Duffy, Jan 07 '19 at 01:40

How to set number of threads as a downstream variable in PBS job queue

1 Answers1