4

I am trying to sumbit some snakemake jobs to a condor cluster with the following command.

snakemake -f TestJob --cluster-config cluster.json -j 30 --cluster condor_submit

where the TestJob rule is specified in the SnakeFile

rule TestJob:
    input:
    output: "test.txt"
    shell:
        "touch test.txt;"

and the cluster-configuration file is the following:

{
    "__default__" :
    {
        "output"    : "workdir/logs/cluster/{rule}.{wildcards}.out",
        "error"     : "workdir/logs/cluster/{rule}.{wildcards}.err",
        "log"       : "workdir/logs/cluster/{rule}.{wildcards}.log"
    },

}

When I do this I obtain the following error:

Building DAG of jobs...
Using shell: /bin/bash
Provided cluster nodes: 30
Job counts:
    count   jobs
    1   TestJob
    1

[Mon Jul  1 11:18:04 2019]
rule TestJob:
    output: test.txt
    jobid: 0


ERROR: on Line 11 of submit file: 

ERROR: Failed to parse command file (line 11).
Error submitting jobscript (exit code 1):
Submitting job(s)
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

Has anyone any idea of what might be happening? Thanks

Simone Meloni
  • 51
  • 1

1 Answers1

0

Snakemake should write the string under the shell directive to a temporary file and pass that file to condor_submit as a command argument. I.e., in your case the temp file would contain only the line touch test.txt; and then snakemake would execute condor_submit temp.sh.

So, what happens if you actually write a file with the only line touch test.txt; and then do condor_submit temp.sh? Just a wild guess as I'm not familiar with condor...

(However, the error message says ERROR: on Line 11 of submit file: so maybe the submitted file has more than 1 line)

dariober
  • 8,240
  • 3
  • 30
  • 47
  • Do you know any way of accessing directly to that submission file? that would be faster to debug – Simone Meloni Jul 02 '19 at 20:41
  • Not sure where the submission file is... However, the test I suggest should be straightforward. Also, try simplifying even more your snakemake execution by removing `--cluster-config cluster.json` – dariober Jul 03 '19 at 08:31