1

I'm trying to run a docker container from snakemake. The jobs run and produce the correct output but when they complete I get

(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

and snakemake tries to remove the files.

Things I've tried to debug this:

  • Adding || true, set +u or exit 0 to the shell command in the rule

  • Running the commands shown in --printshellcmds (they run fine)

  • Putting the docker command in a bash script in 'strict' mode to see if there are any issues (there are not)

  • Printing the actual exit code in the shell: command of the rule and it prints 0

  • Running the R script which is being called in the container directly from the snakemake (it works fine)

  • Adding --user $(id -u):$(id -g) to the docker run command suggested here: Snakemake claims rule exits with non-zero exit code, even with "|| true"?. This fails as the R script in the container has nowhere to write out intermediate files as my current user does not properly exist in the container

Here is the snakefile rule:

rule run_biospyder:
    input: 
        counts = "{dir}/{name}_counts.csv",
        metadata = "{dir}/{name}_metadata.csv",
    output: directory("{dir}/{name}_output")
    params: 
        dockervol = "/usr/src/data"
    shell:
        """
        docker run \
        -v $PWD:{params.dockervol} biospyderpipeline \
        --config-file {params.dockervol}/config.yml \
        --counts {params.dockervol}/{input.counts} \
        --samples {params.dockervol}/{input.metadata} \
        --output {params.dockervol}/{output} \
        --name {wildcards.name}
        """
mark_l
  • 11
  • 3

1 Answers1

0

The issue is a docker permissions issue when mapping volumes. The problem seems to be due to inside container being run as root (I'm just developing) and snakemake tries to write out .snakemake_timestamp as my current user to output directory of the mapped volume (which is owned by root).

I managed to fix it with this: https://denibertovic.com/posts/handling-permissions-with-docker-volumes/

The shell script at the entrypoint creates a user with the same id and runs any commands as that user. This means the mapped volume contents are owned by by user and Snakemake can write to it at the end of the run. This also doesn't fix the user id at the docker build so anyone can use it.

mark_l
  • 11
  • 3