Activating conda environment during gitlab CI

Question

My .gitlab-ci.yml file looks like this:

anomalydetector:
  image: continuumio/miniconda:4.7.10
  stage: build
  tags:
    - docker
  script:
    - conda env create -f environment.yml
    - conda activate my-env
    - pytest tests/.

On Gitlab, this job starts fine, and the logs read

$ conda env create -f environment.yml
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done


==> WARNING: A newer version of conda exists. <==
  current version: 4.7.10
  latest version: 4.7.11

Ok, so I'm using a conda version later than 4.4, so conda activate should work. However, the job fails with the following:

# To activate this environment, use
#
#     $ conda activate my-env
#
# To deactivate an active environment, use
#
#     $ conda deactivate

$ conda activate my-env

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

I have then tried editing my .gitlab-ci.yml file so that there is a command

conda init bash

but then get the message

==> For changes to take effect, close and re-open your current shell. <==

How can I activate my conda environment in the gitlab CI process?

Changing `conda activate my-env` to `source activate my-env` makes it work, though I don't understand why - I thought that for newer versions of conda, `conda activate` should work fine — EuRBamarth, Sep 02 '19 at 09:34
A similar problem with Jenkins is discussed here https://stackoverflow.com/a/57792738/6793245 — orangeInk, Sep 05 '19 at 10:17
I'm struggling to get this working with GitHub Actions. Would love some advice. — chrisdembia, Dec 05 '19 at 06:44

Tommy · Answer 1 · 2019-11-11T16:14:31.520

18

conda init touches the .bashrc file. To reinitialize the shell you can source it:

  - conda create --name myenv
  - conda init bash
  - source ~/.bashrc    # <- !!!
  - conda activate myenv

Whether this is better or worse than source activate myenv is a separate discussion, I guess.

edited Nov 11 '19 at 16:14

answered Nov 11 '19 at 15:47

Tommy

739
8
24

GitLab's [before_script](https://docs.gitlab.com/ee/ci/yaml/#before_script) may offer a better place to add the `conda init`. But `source ~/.bashrc`(or powershell from cweigel's answer) is still required, because `before_script` just prepends the `script` with more commands -- it does not restart the shell. – mike Feb 11 '22 at 23:42

score 8 · Answer 2 · answered Feb 12 '20 at 10:46

Similarly to Tommy's answer, this needs to be done for the Windows Powershell as well. Contrary to bash conda activate myenv does not fail in the powershell. It just has no effect (i.e. the environment is not switched) without calling conda init powershell which makes it even more awkward. Reloading the profile in the powershell is more complicated since there are six of them [1]. I used:

 - conda create --name myenv
 - conda init powershell
 - "if (test-path $PROFILE.CurrentUserAllHosts) { & $PROFILE.CurrentUserAllHosts}"
 - conda activate myenv

Why Conda uses the $PROFILE.CurrentUserAllHosts profile has been asked in an issue [2].

references:

[1] https://devblogs.microsoft.com/scripting/understanding-the-six-powershell-profiles/

[2] https://github.com/conda/conda/issues/8608

score 3 · Answer 3 · answered Jul 06 '21 at 16:25

3

Somehow all of these answers failed me. I ended up using conda run instead of activating an environment. That allowed me to run pytest without activating the environment

- conda run -n <environment-name> python -m pytest <addl-pytest-args>

answered Jul 06 '21 at 16:25

David Kaftan

1,974
1
12
18

For me, adding `source ~/.bashrc`, as also @Tommy said, helped. So `conda init bash`, `source ~/.bashrc`, `conda create ...`, `conda activate ...` all in a `before_script: - |` block – Micha Jun 21 '22 at 14:56

score 1 · Answer 4 · answered Jun 18 '21 at 15:07

Another possibility that you might find more succinct and elegant: source directly the code needed for conda to run run with bash. This also has the effect of adding conda to the PATH, if it's not the case.

This is done with

- source <anaconda_root>/etc/profile.d/conda.sh
- conda activate myenv

(Stolen from https://stackoverflow.com/a/60523131/11343)

Activating conda environment during gitlab CI

4 Answers4

Linked