Stack Overflow
Stack Overflow

Questions tagged [sbatch]

sbatch submits a batch script to SLURM (Simple Linux Utility for Resource Management). The batch script may be given to sbatch through a file name on the command line, or if no file name is specified, sbatch will read in a script from standard input. The batch script may contain options preceded with "#SBATCH" before any executable commands in the script.

sbatch submits a batch script to SLURM (Simple Linux Utility for Resource Management). The batch script may be given to sbatch through a file name on the command line, or if no file name is specified, sbatch will read in a script from standard input. The batch script may contain options preceded with "#SBATCH" before any executable commands in the script.

231 questions
3
votes
1 answer

Chain multiple SLURM jobs with dependency

In a previous question I asked how to queue a job B to start after job A, which is done with sbatch --dependency=after:123456:+5 jobB.slurm where 123456 is the id for job A, and :+5 denotes that it will start five minutes after job A. I now need to…
mhovd
  • 3,724
  • 2
  • 21
  • 47
3
votes
0 answers

Slurm --cpus-per-task command

Hello everyone I'm actually using a soft called RepeatMasker, in this pipeline I can run parallelized job via slurm with the command -pa here is a doc about this command : RepeatMasker -h -pa(rallel) [number] The number of sequence batch…
chippycentra
  • 3,396
  • 1
  • 6
  • 24
3
votes
2 answers

Multiprocessing with python on a single node using slurm

I am trying to run some parallel code on a cluster. The cluster uses slurm and my code is in python. The code uses multiple cores when I run it on my own machine. However, when I try to run the code on the cluster it is extremely slow and does…
John Stenger
  • 31
  • 3
3
votes
1 answer

sbatch+srun: Large amount of single thread jobs

Hello friendly people, my question is rather specific. For more than a week, I am trying to submit thousands of single thread jobs for a scientific experiment using sbatch and srun. The problem is that these jobs may take different amounts of time…
Perhalo
  • 85
  • 8
3
votes
1 answer

Passing a parameter into a slurm script

I am using slurm scripts to run arrays for Matlab computing on a cluster. Each script uses an array to loop over a matlab parameter. 1) Is it possible to create a shell script to loop over another variable? 2) Can I pass variables to a slurm…
Katie Ozgun
  • 53
  • 1
  • 4
3
votes
1 answer

How to make sbatch wait until last submitted job is *running* when submitting multiple jobs?

I'm running a numerical model which parameters are in a "parameter.input" file. I use sbatch to submit multiple iterations of the model, with one parameter in the parameter file changing every time. Here is the loop I use: #!/bin/bash -l for a in…
user222552
  • 95
  • 1
  • 10
3
votes
1 answer

Automatically rerun jobs submitted with sbatch --array upon error

I am submitting jobs in an array. Occasionally one job will error because of a difficult to diagnose gpu memory issue. Simply rerunning the job results in success. What I would like to do is catch this error, log it, and put the job back into…
James Owers
  • 7,948
  • 10
  • 55
  • 71
3
votes
1 answer

How can I send a timeout signal to a wrapped command in sbatch?

I have a program that, when it receives a SIGUSR1, writes some output and quits. I'm trying to get sbatch to notify this program before timing out. I enqueue the program using: sbatch -t 06:00:00 --signal=USR1 ... --wrap my_program but my_program…
Nicolas Loira
  • 31
  • 2
3
votes
1 answer

Specifying SLURM Resources When Executing Multiple Jobs in Parallel

According to the answers here What does the --ntasks or -n tasks does in SLURM? one can run multiple jobs in parallel via ntasks parameter for sbatch followed by srun. To ask a follow up question - how would one specify the amount of memory needed…
StephenChen
  • 711
  • 2
  • 8
  • 20
3
votes
1 answer

Run R code in parallel in a shell without having R file

I've got the following .sh file which can be run on a cluster computer using sbatch: Shell.sh #!/bin/bash # #SBATCH -p smp # partition (queue) #SBATCH -N 2 # number of nodes #SBATCH -n 2 # number of cores #SBATCH --mem 2000 # memory pool for all…
Roshan Mahes
  • 33
  • 3
3
votes
1 answer

How to get IP/hostname of task A's node and pass to task B via SLURM?

I have a (home-grown) cluster network benchmark that I'm trying to run using the SLURM scheduler. The benchmark uses a standard client/server architecture that requires a server IP address (or hostname) argument to the client executable on…
J. Boley
  • 81
  • 1
  • 9
2
votes
1 answer

How to use properly Slurm sbatch and python Multiprocessing

I wan to run a code using multiprocessing in a server with slurm architecture. I want to limit the number of cpus available and that the code creates a child process for every of them. My code could be simplified in this way: def Func(ins) : ### …
Simone Sartori
  • 85
  • 5
2
votes
1 answer

Request maximum number of threads & cores on node via Slurm job scheduler

I have a heterogeneous cluster, containing either 14-core or 16-core CPUs (28 or 32 threads). I manage job submissions using Slurm. Some requirements: It doesn't matter which CPU is used for a calculation. I don't want to specify which CPU a job…
Scott
  • 35
  • 5
2
votes
1 answer

Having issue with slurm. error: "no such file or directory"

I'm trying to run a slurm script using sbatch . However, despite checking my path variable multiple times, i get a file not found error. Moreover I think this has to do with my go environment but I also get a "cannot import absolute path"…
Amen Tesfaye Duki
  • 31
  • 3
2
votes
0 answers

Carriage return not working in slurm output file when running with sbatch

I'm running a job in slurm with sbatch. In job.sh #!/bin/bash #SBATCH -o out.txt python test.py In test.py I'm printing a lot of intermediate information with print(message +' \r', end='') I run the job.. sbatch job.sh However in the output…
user1506145
  • 5,176
  • 11
  • 46
  • 75
1 2
3
…
15 16