Stack Overflow
Stack Overflow

Questions tagged [qsub]

Qsub is a job submission command for high performance computing jobs.

Qsub is a job submission command for high performance computing jobs. It is used by some resource managers and some schedulers, a few of which are TORQUE, PBSPro, OpenPBS, and SGE. It has many different options and is the way that a job (work request) gets queued for consideration in the cluster.

Questions with the "qsub" tag should clearly indicate which resource manager or scheduler is being used.

385 questions
3
votes
1 answer

activate virtual environment using mpirun in interactive qsub environment

I am using the openmdao 0.13 python module in a project. This module is only available as a virtualenv. When I activate the virtual env, it appears to only activate on one node. What could account for this strange behavior? Why are processors on…
kilojoules
  • 9,768
  • 18
  • 77
  • 149
3
votes
1 answer

How to direct qsub output stream to pwd instead of home

After entering a qsub command, the output stream is updated in my home directory as it runs. This causes two problems for me: 1) My /home/brian folder is slow, even writing a few text characters. Utilities with a significant amount of output text…
Brian
  • 1,988
  • 1
  • 14
  • 29
3
votes
1 answer

How does PBS_NODEFILE work in pbs?

For example, I have a file /home/user/nodes, which contains: node1 node2 node3 node4 ... When I try to submit a job like: qsub -v PBS_NODEFILE=/home/user/nodes -l nodes=2 Does it mean that pbs will select 2 nodes from /home/user/nodes list? I…
Brian Yang
  • 61
  • 1
  • 1
  • 9
3
votes
1 answer

Can I use PBS environment variables inside the PBS directives of my script?

Something like: #PBS -t 0-99 #PBS -d "~/$PBS_ARRAYID.output" What I want to do here is to redefine the working directory of each individual job in the job array, using the job's array id. Is this valid code? I need to know before I send to the…
a06e
  • 18,594
  • 33
  • 93
  • 169
3
votes
1 answer

Use variables as argument parameters inside qsub script

I want to pick up a number of models from a folder and use them in an sge script for an array job. So I do the following in the SGE script: MODELS=/home/sahil/Codes/bistable/models numModels=(`ls $MODELS|wc -l`) echo $numModels #$ -S /bin/bash #$…
Sahil M
  • 1,790
  • 1
  • 16
  • 31
3
votes
1 answer

qsub returns error when submitting jobs from node

I have a complex fortran MPI application running under a Torque/Maui system. When I run my application it produces a huge unique output (~20 GB). To avoid that, I produced a RunJob script that breaks up the running in 5 pieces, each producing…
Quim
  • 161
  • 2
  • 7
3
votes
2 answers

script unable to find directories/files when running from qsub cluster script

I'm calling several unix commands and python on a python script from a qsub shell script, meant to run on a cluster. The trouble is that when the script executes, something seems to go awry in the shell, so that directories and files that exist are…
user248237
3
votes
2 answers

Running jobs on a cluster submitted via qsub from Python. Does it make sense?

I have the situation where I am doing some computation in Python, and based on the outcomes I have a list of target files that are candidates to be passed to 2nd program. For example, I have 50,000 files which contain ~2000 items each. I want to…
user2489252
3
votes
1 answer

Capture job id of a job submitted by qsub

I have been looking for a simple way to capture the job ID of a job submitted by qsub. I saw a suggestion was given by providing a name to the job, and using that name. But that's an indirect method. I tried this way but getting an error jobID="qsub…
prabhat
  • 291
  • 3
  • 13
3
votes
6 answers

Syntax for submitting a qsub job without an actual job file?

I would like to submit qsub jobs on the fly without creating discrete job files. So, let's say I have a python script called "get_time.py" that simply reports the time. Instead of making a submission script like this: cat>job.sub<
jake9115
  • 3,964
  • 12
  • 49
  • 78
3
votes
1 answer

linux: job submitted to sge stuck in 'dt' state

I tried adding a job to sqe by qsub. But it seems to be stuck. The state is shown as 'dt'. What could be wrong? I cannot add run any more jobs due to this. How can I remove the job from queue?
User1234321232
  • 517
  • 10
  • 25
3
votes
2 answers

Running R script through qsub

I am trying to run an R script called test.r through qsub. My R script is as follows: #!/usr/bin/Rscript x <- 1 write.csv(x,"test.csv") If in Ubuntu terminal I type R CMD BATCH test.r, then the script behaves as planned; test.csv gets exported in…
user2763361
  • 3,789
  • 11
  • 45
  • 81
2
votes
2 answers

Can I qsub without source .bash_profile

When I submit a bunch of jobs using qsub, the qsub will source the .bash_profile at first. So how can I qsub a job without source .bash_profile firstly.
user1289405
  • 31
  • 1
  • 3
2
votes
1 answer

Is it possible to see qsub job scripts and command line options etc. that will be issued by Snakemake in the dry run mode?

I am new to Snakemake and planning to use Snakemake for qsub in my cluster environment. In order to avoid critical mistakes that may disturb the cluster, I would like to check qsub job scripts and a qsub command that will be generated by Snakemake…
schuma
  • 23
  • 3
2
votes
0 answers

qsub with conda activate?

I would like to submit job with qsub and do conda activate myenv Other posts suggested source activate myenv but didn't work in my case. Log from cluster showed : .../node16/job_scripts/7102106: line 8: activate: No such file or directory Any idea…
Johnny Tam
  • 423
  • 4
  • 16
1 2 3
…
25 26