2

I have to run multiple sbatch slurm scripts for cluster. Say, I have 50 sbatch files and I am running them sequentially in terminal (am using Ubundu) as follows:

sbatch file1.sbatch  
sbatch file2.sbatch
.  
.  
.  
sbatch file50.sbatch

I want to simplify this 50 different commands to run in single command. As I am new working with terminal as well as cluster, I really don't know how to approach this problem. Kindly, suggest me some solution to perform this action (I guess that I need to use some for loop statements, but of which syntax is my doubt). I am completely confused, some relevant documents might also be helpful.

Thank you.

Update: I tried the following script:

#!/bin/bash  
for i in {1..3}  
do   
    sbatch layer$i.sbatch  
done

But, it didn't create as separate jobs. Only single job is submitted as whole. So, this didn't worked for me.

$ ~/Marabou% sbatch call_sbatch.sbatch  
Submitted batch job 4576049

Thanks.

Update:
Following script works:

import os
os.system ("sbatch filename1.sbatch")
os.system ("sbatch filename2.sbatch")

Sumathi Gokul
  • 101
  • 2
  • 8
  • Take a look at `for` loops in `bash`, they should solve you your problem: http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-7.html – Poshi Jul 23 '19 at 09:10
  • I tried this code as follows for just three files.. #!/bin/bash for i in {1..3} do sbatch layer$i.sbatch done But, it didn't create as separate jobs I guess..Only single job is submitted as whole. So, this dint work for me. $ ~/Marabou% sbatch call_sbatch.sbatch Submitted batch job 4576049 Thanks. – Sumathi Gokul Jul 23 '19 at 18:50
  • Update the question with the new information. Code in comments is not readable. But I think you got the right code. Just lacking some semicolons. – Poshi Jul 24 '19 at 09:24
  • I updated the code which I tried, but does not work... – Sumathi Gokul Aug 16 '19 at 14:51
  • Goood! Did you checked the format? – Poshi Aug 19 '19 at 13:01
  • 1
    This sounds like a job for #SBATCH --array, provided each individual job is not dependent on the previous and there is some common iterator within each file – Reuben Zotz-wilson Jun 22 '20 at 11:01

1 Answers1

1

The code you prepared should be run in the interactive shell, not launched to queues.

Just run your code directly:

$ sh call_sbatch.sbatch

or, if your file is executable,

$ ./call_sbatch.sbatch

It should run the inside lines with the loop you created and that loop will send to the queues the jobs you are interested in (in your sample code, three jobs: layer1.sbatch, layer2.sbatch and layer3.sbatch).

Poshi
  • 5,332
  • 3
  • 15
  • 32