3

I understood from the Snakemake docs that the conda directive can take the name of an existing conda environment. However, I am faced with the error EnvironmentNameNotFound: Could not find conda environment. Why can Snakemake not find the existing conda environment? Please see my example below. I am using Miniconda3 and Snakemake v7.19.1, through Ubuntu 20.04 on WSL2.

My Snakefile contains:

rule test:
    output: temp("test.txt")
    conda: "test-env"
    shell: "touch {output}"

The test-env environment was created as follows:

conda create -n test-env -y

Running conda info --envs shows that test-env exists: test-env /home/elh605/miniconda3/envs/test-env.

My snakemake command and the full output:

snakemake -j1 --use-conda


Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job      count    min threads    max threads
-----  -------  -------------  -------------
test         1              1              1
total        1              1              1

Select jobs to execute...

[Tue Jan 10 19:28:50 2023]
rule test:
    output: test.txt
    jobid: 0
    reason: Missing output files: test.txt
    resources: tmpdir=/tmp

Activating conda environment: test-env

EnvironmentNameNotFound: Could not find conda environment: test-env
You can list all discoverable environments with `conda info --envs`.


[Tue Jan 10 19:28:52 2023]
Finished job 0.
1 of 1 steps (100%) done
Removing temporary output test.txt.
Complete log: .snakemake/log/2023-01-10T192850.160679.snakemake.log

I tried using the --conda-prefix option to point Snakemake to my base conda environment, however Snakemake still could not activate test-env. How can I tell Snakemake where to find test-env?

Ezra Herman
  • 53
  • 6
  • This seems more `conda` related than `snakemake`. I can make use of an existing `conda` environment with the current snakemake version under WSL2 without issues. Q: Does `conda activate test-env` work? – euronion Jan 10 '23 at 22:00
  • FWIW, The example you give works on my Ubuntu system with snakemake 7.15 and 7.19. – dariober Jan 11 '23 at 08:42
  • The behavior appears as though the user executing Snakemake's bash script is either not `elh605` or else you don't have Conda configured for bash (e.g., you configured for `zsh`). Have you run `conda init bash` previously? Did this add to `.bashrc` or `.bash_profile`? What happens if you give the absolute path, rather than the environment name, i.e., `conda: /home/elh605/miniconda3/envs/test-env`? – merv Jan 11 '23 at 18:40
  • Thank you all for responding! @euronion, `conda activate test-env` works. Thanks @dariober, good to know it's not a snakemake bug. @merv, I do think I ran `conda init bash` previously. I ran this again, which returned `No action taken`. I also confirmed that `.bashrc` has a conda section. When I give the absolute path to snakemake, the environment is found and used. – Ezra Herman Jan 12 '23 at 07:55
  • 4
    Are you using `bash` as your default shell? Could you post the output of `conda config --show envs_dirs` executed in your shell *and* once from within snakemake, e.g. by putting `shell: "conda config --show envs_dirs"` in your rule above. – euronion Jan 12 '23 at 08:11
  • 3
    You're on the money @euronion. The output included `/home/elh605/miniconda3/envs` in the shell, versus `/home/elh605/miniconda3/envs/snakemake/envs` through Snakemake. My snakefile ran after adding `shell("conda config --add envs_dirs /home/elh605/miniconda3/envs")` to the top of the file, but this is not an elegant solution. Any idea how this might've come about? – Ezra Herman Jan 12 '23 at 09:38
  • No idea how that could happen or how `snakemake` internally changes the `envs_dirs`. Could potentially be a bug. May I instead suggest you export your environment (`conda env export --from-history > test-env.environment.yaml`) and let `snakemake` handle the environment creation and activation (i.e. by specifying `conda: "test-env.environment.yaml"`. Letting `snakemake` handle it instead of reusing and existing environment is also the recommended approach. – euronion Jan 13 '23 at 07:38
  • Thanks @euronion! That is how I use all my other conda environments. However, one of the packages in my workflow cannot be installed with strict channel priority (see my issue [here](https://github.com/ythuang0522/homopolish/issues/57)). As a temporary solution, until the developers (hopefully) provide a fix, I thought I'd use an existing conda environment for this program. This seemed better than asking snakemake to solve all conda environments flexibly. To my knowledge, there isn't an option to solve just one environment flexibly. – Ezra Herman Jan 13 '23 at 09:49
  • Did you try to add the env_dirs in your .condarc? I had the same problem and this solved it – Isy89 Feb 23 '23 at 07:51
  • If I understand your suggestion correctly, that is essentially what my command `shell("conda config --add envs_dirs /home/elh605/miniconda3/envs")` did? – Ezra Herman Feb 23 '23 at 14:14

0 Answers0