LSF job submission - stdout & stderr redirection

Question

I've submitted my job by the following command:

bsub -e error.log -o output.log ./myScript.sh

I have a question: why are the output and errors logs available only once the job ended?

Thanks

Answer 1

LSF doesn't steam the output back to the submission host. If the submission host and the execution host have a shared file system, and the JOB_SPOOL_DIR is in that shared file system (the spool directory is $HOME/.lsbatch by default) then you should see the stdout and stderr there. After the job finishes, the files there are copied back to the location specified by bsub.

Check bparams -a | grep JOB_SPOOL_DIR to see if the admin has changed the location of the spool dir. With or without the -o/-e options, while the job is running its stdout/err will be captured in the job's spool directory. When the job is finished, the stdout/stderr is copied to the filenames specified by bsub -o/-e. The location of the files in the spool dir is $JOB_SPOOL_DIR/<jobsubmittime>.<jobid>.out or $JOB_SPOOL_DIR/<jobsubmittime>.<jobid>.err

[user1@beta ~]$ cat log.sh 
LINE=1
while :
do
  echo "line $LINE"
  LINE=$((LINE+1))
  sleep 1
done

[user1@beta ~]$ bsub -o output.log -e error.log ./log.sh
Job <930> is submitted to default queue <normal>.
[user1@beta ~]$ tail -f .lsbatch/*.930.out
line 1
line 2
line 3
...

Answer 2

According to the LSF documentation the behaviour is configurable:

If LSB_STDOUT_DIRECT is not set and you use the bsub -o option, the standard output of a job is written to a temporary file and copied to the file you specify after the job finishes.