How to use LSB_JOBINDEX in bsub array job arguments in Platform LSF?

Question

I would like to pass LSB_JOBINDEX to as an argument to my script instead of using an environment variable.

This makes my script more LSF agnostic and avoids creating a helper script that uses the environment variable.

However, I was not able to use LSB_JOBINDEX in arguments: it only works as part of the initial command string.

For example, from a bash shell, I use the test command:

bsub -J 'myjobname[1-4]' -o bsub%I.log \
  'echo $LSB_JOBINDEX' \
  '$LSB_JOBINDEX' \
  \$LSB_JOBINDEX \
  '$LSB_JOBINDEX' \
  "\$LSB_JOBINDEX"

and the output of say bsub2.log is:

2 $LSB_JOBINDEX $LSB_JOBINDEX $LSB_JOBINDEX $LSB_JOBINDEX

So in this case, only the first $LSB_JOBINDEX got expanded, but not any of the following ones.

But I would rather not pass the entire command as a single huge string as the 'echo $LSB_JOBINDEX' in this example. I would prefer to just use separate arguments as in a regular bash command.

I've also tried to play around with %I but it only works for -o and related bsub options, not for the command itself.

Related: Referencing job index in LSF job array

Tested in LSF 10.1.0. Related documentation: https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_admin/job_array_cl_args.html

Answer 1

bsub will add single quotes around the arguments if the argument starts with $. For example. If the bsub command line is

bsub command -a $ARG1 -b $ARG2

Then bsub will add quotes to the arguments to the 2nd and 4th parameters. The command is stored like this

command -a '$ARG1' -b '$ARG2'

One way to prevent this is to put the commands in a script. Like this:

$ cat cmd
echo $LSB_JOBINDEX
echo "line 2"
echo $LSB_JOBINDEX

Then run your job like this:

$ bsub -I < cmd
Job <2669> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0
line 2
0

Note that the -I is not needed. Its just so you can see the job output on the bsub's stdout.

EDIT

OK. Looks like this works. But its not really a serious answer since it's so ugly. The thing is that bsub will surround the argument with single quotes if the argument starts with $. So the strategy is to find some way to make sure that the first character in the argument isn't a $. One way is to put any character other than $ as the first character of the argument. Follow it by a backspace literal, followed by the $. Note that it needs to be the actual backspace character, not ^ followed by H. Use ctrl-v followed by a ctrl-h to get the literal appended to the command line.

$ bsub -I echo "x^H\$LSB_JOBINDEX" "x^H\$LSB_JOBINDEX"
Job <2686> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0 0

EDIT2

A tab literal also works. Not that its much better.

$ bsub -I echo "       \$LSB_JOBINDEX" "       \$LSB_JOBINDEX"
Job <2687> is submitted to default queue <normal>.
<<Waiting for dispatch ...>>
<<Starting on hostA>>
0 0