3

Context : I'm trying to setup a gitlab CI/CD that tests my build and runs my pytest tests when pushing my code.

Problem : When I push my code, the CI/CD job fails saying :

/bin/bash: line 55: pytest: command not found
ERROR: Job failed: exit code 1

Question : How do I get rid of the error and how do I setup properly my gitlab CI/CD ?

Details : I've (partially) followed this guide, and I've made a .gitlab-ci.yml file like this :

image: continuumio/miniconda3:latest

testbuild :
  stage: build
  script:
    - conda create --name test_env --file requirements.txt
    - source activate test_env
    - python setup.py install

tests:
  stage: test
  script:
    - cd tests && pytest .

My project architecture :

$ tree -L 1
project
├── package1/
├── package2/
├── data/
├── out/
├── __pycache__
├── requirements.txt
├── setup.py
└── tests/

My requirements.txt (stripped from a lot of useless stuff, for the readers convenience sake), that has been created with the command conda list -e :

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: linux-64
scikit-learn=0.20.0=py36h4989274_1
scipy=1.1.0=py36hfa4b5c9_1
# ...
setuptools=40.4.3=py36_0
pip=10.0.1=py36_0
py=1.7.0=py36_0
pytest=3.9.1=py36_0
python=3.6.6=h6e4f718_2
wheel=0.32.1=py36_0
021
  • 323
  • 2
  • 17
  • What are the contents of `requirements.txt`? – darthbith Oct 25 '18 at 22:16
  • Oh yeah I knew I forgot something, sorry. It has been edited. I did not put the whole list as it felt a lot of content for not much, but I tried to put the dependencies that mattered (first of all, pytest). – 021 Oct 26 '18 at 08:07
  • Probably the conda env activation is unset between stages, try to reactivate it in the test stage: `cd tests && source activate test_env && pytest .` – hoefling Oct 26 '18 at 08:19
  • I did as you wrote and I got something different : `$ cd tests && source activate test_env &&pytest . Could not find conda environment: test_env You can list all discoverable environments with conda info --envs.` – 021 Oct 26 '18 at 08:27
  • 1
    Looks like gitlab does not preserve the env between stages. Take a look at [GitLab CI preserve environment between build stages](https://stackoverflow.com/questions/38869929/gitlab-ci-preserve-environment-between-build-stages). – hoefling Oct 26 '18 at 22:12

1 Answers1

3

I've changed my .gitlab-ci.yml to :

image: continuumio/miniconda3:latest

testbuild :
  stage: build
  script:
    - conda create --name test_env --file requirements.txt
    - source activate test_env
    - python setup.py install
    - cd tests && pytest .

Regrouping both tests and testbuild in the same part. It now works, it installs everything and runs the tests, though it feels like a bad way to do it, because I'm not doing the separation anymore.

As hoefling said in the comments, the problem is that gitlab does not preserve the environment between stages. If you really do want to separate these two, look at this : GitLab CI preserve environment between build stages

021
  • 323
  • 2
  • 17
  • 1
    I agree with this solution (as I just spent a lot of time coming to it by myself!). You could also do conda env create -f but I feel conda create is currently more flexible. You can actually do offline with "conda create" vs. "conda env create" (assuming you had the packages required in your base conda repo). – ryanskeith Mar 13 '19 at 00:35