Stack Overflow
Stack Overflow

Questions tagged [pbs]

PBS stands for portable batch system, and describes a family of software products for high performance computing.

PBS stands for portable batch system, and describes a family of software products for high performance computing. The software is a resource manager that is used to manage jobs, including their submission, running, and basic managing. They are often used in UNIX cluster environments and can often be used with schedulers.

Modern PBS daemons are descendants from OpenPBS; two notable descendants are TORQUE and PBSPro. TORQUE is an open source product and is maintained by Adaptive Computing. More information can be found on its wikipedia page. Documentation for TORQUE can be found on Adaptive's website. PBSPro is a commercial product that is developed by Altair Engineering. PBSPro's user guide can be found here.

428 questions
4
votes
2 answers

PBS script -o file to multiple locations

Sometimes when I run jobs on a PBS cluster, I'd really like the joblog (-o file) in two places. One in the $PBS_O_WORKDIR for keeping everthing together and one ${HOME}/jobOuts/ for greping/awking/etc... Doing a test from the command line works with…
caddymob
  • 317
  • 1
  • 3
  • 10
4
votes
1 answer

How fast can one submit consecutive and independent jobs with qsub?

This question is related to pbs job no output when busy. i.e Some of the jobs I submit produce no output when PBS/Torque is 'busy'. I imagine that it is busier when many jobs are being submitted one after another, and as it so happens, of the jobs…
qAp
  • 1,139
  • 2
  • 12
  • 26
3
votes
1 answer

List of job_state values for torque

What are the possible values for job_state in Torque? I looked in the related man pages have yet to find a complete list.
sutee
  • 12,568
  • 13
  • 49
  • 61
3
votes
2 answers

Python script on PBS fails with error =>> PBS: job killed: ncpus 37.94 exceeded limit 36 (sum)

I get the error mentioned in the title when I run a python script (using Miniconda) on a PBS scheduler. I think that numpy is doing some multithreading/processing but I can't stop it from doing so. I added these lines to my PBS script: export…
Yair Daon
  • 1,043
  • 2
  • 15
  • 27
3
votes
0 answers

Using parameters from BASH file name as arguments for Julia script on cluster

This is an extension of a previous question I asked: Using name of BASH script as input argument My goal is to write a BASH script which takes the arguments from the file's name and uses them as inputs for a Julia code I'm writing, and then submit…
Joshuah Heath
  • 663
  • 1
  • 5
  • 20
3
votes
1 answer

Starting Jupyter notebook on a node of my cluster (High Performance Computation or HPC facility )

I wanted to run jupyter notebook on a node of our cluster, NOT on the login node. I could remotely run the jupyter notebook on the login node, but it would unnecessarily slow down the cluster usage. Please guide me how I can start the jupyter…
deltasata
  • 377
  • 1
  • 4
  • 21
3
votes
1 answer

PBS: GNU parallel: hosts allocated vary, multi CPU job, multiple jobs to some hosts

With PBSpro I can request resources to run my job. My parallel cluster job boils down to running the same file multiple times, each time with a different index / job ID. Each task spawns its own sub-processes and each task in total uses 4 CPUs. This…
Jurgen Strydom
  • 3,540
  • 1
  • 23
  • 30
3
votes
0 answers

Monitor HPC/PBS/torque usage

I am trying to figure out how best to monitor usage of our HPC resources. Specifically, I am trying to identify cpu usage, disk space consumed, and number of jobs run by group. The pbs format allows the "-W" group_list flag to identify the group…
nietzschemouse
  • 87
  • 7
3
votes
2 answers

Setting PBS/Torque/qsub parameters in script via command line arguments

I want to be able to easily change how many nodes, ppn, etc I submit to qsub via script. That is, I want run somthing like this: qsub script.sh --name=test_job --nodes=2 --ppn=2 --arg1=2 With a script like the following: #/bin/bash #PBS -N…
AKW
  • 857
  • 8
  • 14
3
votes
2 answers

When using qsub to submit jobs, how can I include my locally installed python packages?

I have an account on a supercomputing cluster where I've installed some packages using e.g. "pip install --user keras". When using qsub to submit jobs to the queue, I try to make sure the system can see my local packages by setting "export…
user1634426
  • 563
  • 2
  • 5
  • 12
3
votes
1 answer

How to get PBS/Torque/Slurm job completion status in a Python script

I am writing a python script which prepares input files for hundreds of jobs (say job.id = 1 to job.id = 1000, where job.id is a self-assigned id) and then submit them on a cluster for execution. Each job has 3 stages, s1, s2 and s3, where s2 is…
UCU110
  • 413
  • 1
  • 5
  • 13
3
votes
1 answer

Understanding the -t option in qsub

The documentation is a bit unclear on exactly what the -t option is doing on a job submission using qsub http://docs.adaptivecomputing.com/torque/4-0-2/Content/topics/commands/qsub.htm From the documentation: -t Specifies the task ids of a job…
David Parks
  • 30,789
  • 47
  • 185
  • 328
3
votes
1 answer

Torque PBS: Specifying stdout file name to be the job id number

By default, output from a submitted job to a Torque queue will be saved to a file named like job_name.o658392. What I want to do, using that example, is to name the output file 658392.job_name.log instead. I know I can specify the name of the…
jonaslb
  • 179
  • 9
3
votes
1 answer

How does PBS_NODEFILE work in pbs?

For example, I have a file /home/user/nodes, which contains: node1 node2 node3 node4 ... When I try to submit a job like: qsub -v PBS_NODEFILE=/home/user/nodes -l nodes=2 Does it mean that pbs will select 2 nodes from /home/user/nodes list? I…
Brian Yang
  • 61
  • 1
  • 1
  • 9
3
votes
1 answer

PBS torque email variable

Here is an example of pbs that I use: #!/bin/bash #PBS -S /bin/bash #PBS -N myJob #PBS -l nodes=1:ppn=4 #PBS -l walltime=50:00:00 #PBS -q route export MYMAIL=mytestmail@testmail.com #PBS -m ae #PBS -M mytestmail@testmail.com ./script1.sh echo…
Yohann Morille
  • 153
  • 7
1 2 3
…
28 29