How to use lsf.yaml with snakemake?

Question

I would like to use snakemake with LSF.
I follow this url .

My Snakefile contain:

rule all:
input:
    "foo.txt",
    "file.out"

rule foo:
    input: "foo.txt"
    output: "bar.txt"
    shell:
        "set +o pipefail; grep bar {input} > {output} "

rule bar:
    input: "bar.txt"
    output: "file.out"
    shell:
        "echo blah > {output}"

In the same path, I have the file lsf.yaml. The file contain:

__default__:
  - "-q medium"
foo:
  - "-q short"

It's run okay when I run it with the command :

snakemake -j 1

It failed when I tried to test it with lsf. I run:

snakemake -j 1 --cluster "bsub"

I got:

Building DAG of jobs... Using shell: /usr/bin/bash Provided cluster
nodes: 1 Job counts:
count jobs
1 all
1 bar
1 foo
3

[Thu Jun 18 15:47:21 2020] rule foo:
input: foo.txt
output: bar.txt
jobid: 2

Memory reservation is (MB): 1024 Memory Limit is (MB): 1024 training:
No such queue. Job not submitted. Error submitting jobscript (exit code 255):

Shutting down, this might take some time. Exiting because a job
execution failed. Look above for error message Complete log:
/home/student6/snakemake_test/.snakemake/log/2020-06-18T154721.snakemake.log

In the log file I have the same message as above plus:

Error submitting jobscript (exit code 255):

How can I make snakemake run with LSF?
Thanks.

It appears snakemake-lsf-profile requires specifying [--profile](https://github.com/Snakemake-Profiles/snakemake-lsf#usage), which is missing in your command. However, your error message seems to suggest this profile is somehow identified and used. What is the lsf queue name used here? — Manavalan Gajapathy, Jun 18 '20 at 16:10

score 1 · Answer 1 · answered Sep 29 '20 at 07:31

1

The issue here is you are using the --cluster option instead of --profile. So, if you named your profile bsub, following the install instructions here, then this should fix the issue for you

$ snakemake -j 1 --profile "bsub"

answered Sep 29 '20 at 07:31

Michael Hall

2,834
1
22
40

score 0 · Answer 2 · answered Jun 20 '20 at 13:24

0

No such queue. Job not submitted. Error submitting jobscript (exit code 255):

perhaps this is the key error message. Do the configured queues exist in your LSF cluster?

__default__:
  - "-q medium"
foo:
  - "-q short"

Use bqueues to list the available queues. short is one of the pre-configured queues, but medium isn't.

answered Jun 20 '20 at 13:24

Michael Closson

902
8
13

both queues defined in lsf – user1980099 Jun 21 '20 at 06:01

How to use lsf.yaml with snakemake?

2 Answers2