It was my understanding that the virtual env created by poetry would by default not give access to any packages outside that env, not even the ones installed in the system directory (such a feature is being added in https://github.com/sdispater/poetry/issues/1393 which I think implies the default behavior is the opposite). Developing altair_recipes https://github.com/piccolbo/altair_recipes/ I had a couple of instances where I forgot to add a new dep to pyproject.toml, tests succeeded locally but failed on travis on loading that package. The situation is promptly fixed by adding the dependency with poetry add. Most recent example, I started using hypothesis for some tests, forgot to add it, everything worked locally, went for release, fail. Added as dev dep, pass local, pass remote, done ( see https://travis-ci.org/piccolbo/altair_recipes build 102 and 103). The only theory I have is that the local testing was able to pick up the system installation of that package. The tests are run with poetry run py.test With the goal of having local tests match tests done in a CI setting in a fresh environment, what can I do to have the local env best match the CI env? And is isolation from the system installation really the problem? Thanks
- 1,305
- 7
- 17
-
Are you setting `PYTHONPATH`? – jpyams Oct 08 '19 at 15:24
-
@jpyams No and 15 more characters to satisfy the SO AI – piccolbo Oct 10 '19 at 17:28
-
Are you running your tests with `poetry run`? – jpyams Oct 10 '19 at 20:22
-
Yes and 12 more characters to appease SO gods. – piccolbo Oct 11 '19 at 23:44
2 Answers
The only theory I have is that the local testing was able to pick up the system installation of that package.
Given my personal experience with packaging in python, I'd assume that this is the underlying reason. Now, there are a long-term and a short-term solution to this problem:
Long-term you should adopt the mindset of never installing python packages that are tied to a single project system wide. This includes dev-dependencies like pytest or in your case hypothesis - uninstall them. poetry is one of the very few python packages that are relevant system wide and should be installed as such. Additionally a side note, this is why the recommended way to install poetry is not pip but get-poetry.py.
Short-term, you can set up your virtual environment by hand with virtualenv, which blocks global site-packages by default (but it is likely that the envs that poetry build have the same default setting). To enforce execution of locally installed packages you can either set the execution path before running tests:
$ PATH=$(dirname $(poetry run which python)) pytest
If you have activated the virtual env (which you can do by running poetry shell) you can also run the tools as python modules:
(env)$ python -m pytest
- 17,706
- 5
- 83
- 99
-
Setting `PYTHONPATH` doesn't exclude system packages, it just adds other paths to the front of your `sys.path` – jpyams Oct 11 '19 at 19:19
-
1
-
Sorry, I came here because poetry points to here to get support. Your suggestions may be good, but I am looking for poetry support, not to use other programs or remove other packages. SO is a very noisy place to use as support channel. – piccolbo Oct 11 '19 at 21:30
-
I did try to answer your question. The thing is, you cant solve that problem with poetry alone, poetry cant control where your system looks for executables. Do my suggestions not work? – Arne Oct 11 '19 at 21:38
I am experiencing bug https://github.com/sdispater/poetry/issues/571 I need 8 more characters to post this. Clearly SO and don't share the same opinion on conciseness.
- 1,305
- 7
- 17