Problems with command substitution (variable is empty)

Question

Having some trouble with command substitution in a shell script. I've used this script before on another system, so not sure what's going on here. Seems like no matter how I enclose the commands to set the variable, I'm getting an empty variable. I've tried many different ways to enclose them ($(), backticks, quotes). If it helps, I'm trying to submit an array job.

#!/bin/bash
#BSUB -q shared
#BSUB -W 23:55
#BSUB -M 51200
#BSUB -J windowCovR1
#BSUB -R "span[hosts=1]"

mem=$LSB_JOBINDEX
sample=`head -n "$mem" bednames.txt | tail -1 | awk '{print $2}'`
eval ${sample}
echo ${sample}

$mem is being set properly, it's something with the sample line. The echo command prints an empty line.

[I'm trying to put a command in a variable, but the complex cases always fail!](http://mywiki.wooledge.org/BashFAQ/050) — fedorqui, Jul 06 '16 at 14:45
I don't think that really addresses this problem...this has worked before for me (on a different system), so not sure what is going on now. — Ethan, Jul 06 '16 at 15:03
Perhaps `bednames.txt` is empty? A better way to get line `N` from the file might be `sample=$(awk -vlineno=${mem} 'NR == lineno { print $2 }' bednames.txt)`... — twalberg, Jul 06 '16 at 18:16
It's not empty...I ended up doing this with a loop which wasn't ideal. Will update if I solve this. — Ethan, Jul 07 '16 at 13:19

score 0 · Answer 1 · edited Jul 03 '19 at 02:19

0

Your approach of "try[ing] backticks" does not seem to be correct.

If the result of the substitution is empty, it's because that's the variable's value. Have you tried debugging the command whose output sets that variable via the terminal?

edited Jul 03 '19 at 02:19

cs95

379,657
97
704
746

answered Jul 06 '16 at 17:31

Bruce David Wilner

463
2
4

score 0 · Answer 2 · answered Jul 07 '16 at 19:39

Since you're using $LSB_JOBINDEX, you probably want an array job. The job name should include the start and end parameters of the array. e.g.,

#BSUB -J windowCovR1[1-100]

This assume that there are 100 command lines in the file bednames.txt. Without the array bounds, $LSB_JOBINDEX will always be zero. head -n 0 will give empty string.

I tried this simple example and it works ok for me.

 [mclosson@host ~]$ cat tmp.sh
 #!/bin/bash
 #BSUB -q normal
 #BSUB -W 23:55
 #BSUB -M 10
 #BSUB -J windowCovR1[1-2]
 #BSUB -R "span[hosts=1]"

 mem=$LSB_JOBINDEX
 echo "$$ mem $mem" >> /tmp/LOG
 sample=`head -n "$mem" bednames.txt | tail -1 | awk '{print $2}'`
 echo "$$ ${sample}" >> /tmp/LOG

 [mclosson@host ~]$ cat bednames.txt
 line11 line12
 line21 line22

 [mclosson@host ~]$ bsub < tmp.sh
 Job <805> is submitted to queue <normal>.

 [mclosson@host ~]$ cat /tmp/LOG
 7979 mem 1
 7979 line12
 7991 mem 2
 7991 line22

Another small point. If this isn't a parallel job then -R "span[hosts=1]" isn't needed.

score -1 · Answer 3 · answered Jul 06 '16 at 17:53

-1

Your command to assign the value of sample is correct. Working for me as well on command line. However, you are using the same in shell script. So for that try using below :-

sample=head -n "$mem" bednames.txt | tail -1 | awk '{print \$2}'

\ before $2. It might happen that when you are using without \ , the script checks for second command line argument.

answered Jul 06 '16 at 17:53

Lalit Somnathe

111
6

Within single quotes, there is no need to escape the `$`, and doing so will in fact cause `awk` to complain about a syntax error. – twalberg Jul 06 '16 at 20:45

Problems with command substitution (variable is empty)

3 Answers3