How to automate the LSF waiting based on job name in perl

Question

I have a perl code where I am submitting few jobs at once in parallel via LSF bsub command and once all these jobs finish want to submit a final job.

For example I have these three bsub commands where first two bsub commands submits jobs t1 and t2 and third command checks whether t1 and t2 are finished or not and wait on them with -w argument.

   system(" bsub -E "xyz" -n 1 -q queueType -J t1 sleep 30")
    system("bsub -E "xyz" -n 1 -q queueType -J t2 sleep 30")
    system("bsub -E "xyz" -n 1 -q queueType -J t3 -w "done(t1)&&done(t2)" sleep 30")

So for automating -w argument I have this

    my $count=2;
    my $i; 
   system(" bsub -E "xyz" -n 1 -q queueType -J t3 \"foreach my $i (0..$count) {print " done(t_$i)&&";}\" sleep 30 ")

I get this error:

sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `bsub -E "/pkg/ice/sysadmin/bin/linux-pre-exec" -n 1 -q short -J t3 -w "foreach  (0..7) {print \"done(t)&&\";}" sleep 30'

EDIT: Yes I am using system command to submit these jobs from perl

Are you running your Perl code with `sh`? If so, why? On the other hand, that third line doesn't look like Perl. What is going on? — melpomene, Aug 04 '19 at 21:15
Or do you mean to run `bsub` out of a Perl script, attempted with "_So for automating..._" ? — zdim, Aug 04 '19 at 21:17

melpomene · Accepted Answer · 2019-08-04T21:48:47.647

If you want to generate the done(...)&&done(...) string dynamically, you can use

my $count = 7;
my $done_all = join '&&', map "done(t$_)", 1 .. $count;

That is, for each number in the range 1 .. 7, produce a string "done(t$_)", which gives a list "done(t1)", "done(t2)", ... "done(t7)". The elements of this list are then join'd together with a separator of &&, producing "done(t1)&&done(t2)&&...&&done(t7)".

To run an external command, you can use system. Passing a list to system avoids going through the shell, which avoids all kinds of nasty quoting issues:

system('bsub', '-E', 'xyz', '-n', '1', '-q', 'queueType', '-J', 't3', '-w', $done_all, 'sleep', '30');
# or alternatively:
system(qw(bsub -E xyz -n 1 -q queueType -J t3 -w), $done_all, qw(sleep 30));

Your code tries to pass Perl code to bsub, but that won't work. You have to generate the command string beforehand and pass the result to bsub.

How to automate the LSF waiting based on job name in perl

1 Answers1