Stack Overflow
Stack Overflow

Questions tagged [qsub]

Qsub is a job submission command for high performance computing jobs.

Qsub is a job submission command for high performance computing jobs. It is used by some resource managers and some schedulers, a few of which are TORQUE, PBSPro, OpenPBS, and SGE. It has many different options and is the way that a job (work request) gets queued for consideration in the cluster.

Questions with the "qsub" tag should clearly indicate which resource manager or scheduler is being used.

385 questions
9
votes
2 answers

how to get the job id from qsub

I am using a cluster. I will use qsub command to distribute my job, what I want is I can get back the job id so that I can monitor the job. Basically, I want some thing like this: #!/bin/bash JOBID=$( qsub job1 ) # Monitoring base on $JOBID I found…
sflee
  • 1,659
  • 5
  • 32
  • 63
8
votes
1 answer

Exclude certain nodes when submitting jobs with qsub / torque?

When submitting batch jobs with qsub, is there a way to exclude a certain node (by hostname)? Something like # this is just a pseudo command: qsub myscript.sh --exclude computer01
jdm
  • 9,470
  • 12
  • 58
  • 110
8
votes
2 answers

Argument passing in .sh scripts

I have a shell script foo.sh which is a qsub job with content: #!/bin/bash -l #$ -S /bin/bash #$ -N $2 echo $1 I would like to pass two arguments. If I call qsub foo.sh a b the first argument gets correctly processed and echoed to…
user1137731
  • 133
  • 1
  • 2
  • 6
8
votes
2 answers

Getting the exit code from a process submitted with qsub on Sun Grid Engine

I would like to submit jobs via qsub on Sun Grid Engine (now: Oracle Grid Engine?). I do not wish to use the -sync yes option or qrsh, because I want my controlling program to be single-threaded and able to launch many jobs at a time. These…
Brian
  • 690
  • 1
  • 7
  • 18
7
votes
3 answers

Making qsub block until job is done?

Currently, I have a driver program that runs several thousand instances of a "payload" program and does some post-processing of the output. The driver currently calls the payload program directly, using a shell() function, from multiple threads. …
dsimcha
  • 67,514
  • 53
  • 213
  • 334
7
votes
4 answers

Running qsub with anaconda environment

I have a program that usually runs inside a conda environmet in Linux, because I use it to manage my libraries, with this instructions: source activate my_environment python hello_world.py How can I run hello_world.py in a high computer that works…
Ushuaia81
  • 495
  • 1
  • 6
  • 14
7
votes
3 answers

how to automatically run a bash script when my qsub jobs are finished on a server?

I would like to run a script when all of the jobs that I have sent to a server are done. for example, I send ssh server "for i in config*; do qsub ./run 1 $i; done" And I get back a list of the jobs that were started. I would like to automatically…
David LeBauer
  • 31,011
  • 31
  • 115
  • 189
7
votes
1 answer

os.system vs subprocess in python on linux

I have two python scripts. The first script calls a table of second scripts in which I need to execute a third party python script. It looks something like this: # the call from the first script. cmd = "qsub -sync y -b -cwd -V -q long -t 1-10 -tc…
alex_milhouse
  • 891
  • 1
  • 13
  • 31
6
votes
2 answers

How does MATLAB handle file change while the file is being executed?

Assume that you start running the script. What happens when you change that file when it is being executed? It seems that MATLAB takes a copy of the file and then starts executing it. I want to make sure that I am right. That said, I want to run a…
Mohammad Moghimi
  • 4,636
  • 14
  • 50
  • 76
6
votes
2 answers

can i delete a shell script after it has been submitted using qsub without affecting the job?

I want to submit a a bunch of jobs using qsub - the jobs are all very similar. I have a script that has a loop, and in each instance it rewrites over a file tmpjob.sh and then does qsub tmpjob.sh . Before the job has had a chance to run, the…
miz
  • 185
  • 2
  • 10
6
votes
1 answer

Torque job with dependencies on job array

I'm trying to submit a torque job that is dependent on an array completing. FIRST=`qsub -q hep -t 1-5 foo.sh` qsub -q hep -W depend=afterok:$FIRST bar.sh The FIRST job array submits and completes just fine but the second job bar.sh just holds…
J. Vasquez
  • 161
  • 8
6
votes
3 answers

How to use qdel all to remove only idle jobs

I am having an issue where if I have 2000 jobs queued and try to delete them with qdel all, it'll keep trying to delete the running jobs first. This means I have to wait a long time for the jobs to get deleted because removing from the Running list…
user2763361
  • 3,789
  • 11
  • 45
  • 81
6
votes
3 answers

How can I use a pipe or redirect in a qsub command?

There are some commands I'd like to run on a grid using qsub (SGE 8.1.3, CentOS 5.9) that need to use a pipe (|) or a redirect (>). For example, let's say I have to parallelize the command echo 'hello world' > hello.txt (Obviously a simplified…
David Robinson
  • 77,383
  • 16
  • 167
  • 187
5
votes
8 answers

SGE script: print to file during execution (not just at the end)?

I have an SGE script to execute some python code, submitted to the queue using qsub. In the python script, I have a few print statements (updating me on the progress of the program). When I run the python script from the command line, the print…
miz
  • 185
  • 2
  • 10
5
votes
1 answer

PBS jobs inter-dependency: one job starts, cancel others

I would like to submit a simulation to several queues on my cluster. As soon as one queue would start it, it would be cancelled on the others. I understand it is potentially ill-defined as several jobs could start at the same time on several…
user1824346
  • 575
  • 1
  • 6
  • 17
1
2
3
…
25 26