0

I have a batch job which looks like this

sbatch --wrap "perl test.pl file1 file2"

sbatch --wrap "perl test.pl file3 file4"

sbatch --wrap "perl test.pl file5 file6"

sbatch --wrap "perl test.pl file7 file8"

& the list goes on till

sbatch --wrap "perl test.pl file49 file50"

How can I run individual jobs sequentially? Thanks,

JTh
  • 13
  • 2
  • 3
    I guess it depends on what a "job" is, but isn't that what your batch is doing? – ForeverZer0 Jul 19 '18 at 02:39
  • 2
    1-2-3-4 and on until 49-50 is already sequential. What can we help you with? – Ken White Jul 19 '18 at 02:41
  • Actually, when I paste them on the command line in this format (or put them together in a file), they all run in parallel. – JTh Jul 19 '18 at 02:56
  • Did you happen to read the help for SBATCH? That is what it was designed to do. – Squashman Jul 19 '18 at 03:13
  • I have never used sbatch but did a quick Google Search with your keywords and found this. https://chuckaknight.wordpress.com/2016/08/02/slurm-scheduling-so-they-run-sequentially/ – Squashman Jul 19 '18 at 03:22
  • @KenWhite, the problem is that once on the queue, they can all be started by the system at once and run simultaneously. – Poshi Jul 19 '18 at 07:27
  • @Squashman Thanks. I first tried the answer posted by Poshi below and that worked perfectly. – JTh Jul 19 '18 at 20:30
  • @JTh, then why did you use the `batch-file` tag? If you used Poshi's code below then your question has nothing to do with the `batch-file` tag. – Squashman Jul 19 '18 at 20:32
  • @Squashman Sorry, that was a mistake. I had meant to write "batch job". – JTh Jul 19 '18 at 20:46

1 Answers1

0

You must make one script dependent on the finalization of the previous one:

JOBID=$(sbatch --wrap "perl test.pl file1 file2" | awk '{print $4}')
for N in {3..49..2}
do
    JOBID=$(sbatch -d afterok:$JOBID --wrap "perl test.pl file$N file$(($N+1))" | awk '{print $4}')
done

BTW, the singleton approach suggested in a comment is also a good approach, and frees you from the JobID management harrasement.

Poshi
  • 5,332
  • 3
  • 15
  • 32
  • I think they might be running this on Windows considering the `batch-file` tag they used. They may not have bash shell installed. – Squashman Jul 19 '18 at 12:27
  • Well, at least he can grab the idea. How he implements the loop is completely up to him. Just putting all the commands one after the other will do. Or using the singleton approach, which will be even easier for this specific task. – Poshi Jul 19 '18 at 12:59
  • @Poshi your solution worked perfectly. Thanks so much. – JTh Jul 19 '18 at 20:26
  • I know ;-) I do that kind of things on a daily basis. Now, accepting the answer would be a good thing :-) – Poshi Jul 19 '18 at 21:01