0

I am writing a long workflow for scientific data processing that uses Snakemake. Along the workflow, there are multiple scripts that use different environments. The folder looks something like this:

package
|-- scripts
|   |
|   -- script1.py
|   -- script2.py
|-- tests
|   |
|   -- test1.py
|   -- test2.py
|-- envs
|   |
|   -- env1.yml
|   -- env2.yml

test1 uses conda environment in env1.yml to test script1 and so on. Currently, I can run individual tests with:

conda activate env1 && python -m pytest tests/test1.py && conda deactivate

Is there a way I can use pytest to pair the tests with the right environment automatically? Right now I am thinking about creating a bash script to activate env and run the test with one line per test. Alternatively, I was also looking into multiple pytest configurations as described in this SO link.

I was hoping to this natively with pytest instead of bash scripts if that is possible.

spo
  • 304
  • 1
  • 7
  • 1
    I don't think it's possible to do it from Pytest, by the time Pytest is running, you are already in a specific Python environment. I think your bash script approach makes more sense. Using Pytest markers can make selecting the tests for each environment easier. – Matias B Aug 31 '23 at 18:37
  • 2
    I've been on teams that used tox to do this: https://tox.wiki/en/4.11.0/ Tox basically spins up virtualenvs on the fly and runs your tests within the virtualenv. I've never used it alongside conda but I assume as long as you have a requirements file it should work the same. – commanderdata Aug 31 '23 at 18:53
  • @commanderdata, I also considered tox. But to my understanding, tox runs all your tests in all the specified environments? Is there a way to pair an environment with a test? – spo Aug 31 '23 at 19:46
  • @spo nope, you can run whatever subset you would like (as long as pytest supports it). You just specify the tests you would like to run from your pytest command within the commands section of your tox environment. Here's the user guide: https://tox.wiki/en/latest/user_guide.html – commanderdata Sep 01 '23 at 17:48

0 Answers0