11

Say I have a project named Foo, and and want to install requests package locally for this project. What I am expecting is some structure similar to this:

Foo/
|-main.py
|-requirements.txt
|-README.md
|-python_modules/
|-|-requests
...

And I can do this by pip install -r requirments.txt -t ./python_modules/, however, this doesn't work properly because there are no __init__.py under python_modules/ so programs won't automatically import every packages in python_modules.

On the other hand, npm install does this very well.

So my question is, how to let pip work the same as npm does?

PS: I know there are other conventions using virtualenv or pythonbrew, but I still want to ask this question.

mingyc
  • 526
  • 5
  • 13
  • `virtualenv`. Python does not have this possibility by default. – TheHippo May 04 '13 at 15:12
  • it would work if you added the `python_modules` directory to `sys.path` in your main. – mata May 04 '13 at 15:17
  • 1
    As @mata said, you can change `sys.path` or just export `PYTHONPATH` environment variable. Like: `$ export PYTHONPATH=$PWD/python_modules` and you are set – Hugo Lopes Tavares May 04 '13 at 20:53
  • Unfortunately, that $PWD is set when you run the command, so you can't simply do this in a dotfile and adopt it as a convention that "just works" for all of your projects. I suppose you could create a bootstrap script but it would be nice if this was transparent. – Koobz Jul 13 '13 at 06:04
  • 4
    But you can add the string `./python_modules` to the `PYTHONPATH` variable, so that when you run `python` from the project's root folder it will reference the local `python_modules` folder. This cross-project solution works for me on Linux, Windows and Mac. – ehremo Apr 28 '15 at 11:26
  • @ehremo please add an answer with this info, so I can upvote :D – alexandrul Oct 21 '16 at 09:44
  • @Koobz if you wanted to put the export mentioned you could always set it in `PROMPT_COMMAND`. – Steve Buzonas Jun 09 '17 at 22:16

3 Answers3

1

The usual solution to this problem in the Python world is to use virtualenvs or, even better, a wrapper like pipenv. If you install pipenv, you should be able to create a new virtualenv with a simple pipenv install:

[user@host Foo]$ pipenv install
Creating a virtualenv for this
project… ⠋Using base prefix '/usr' New python executable in
/home/user/.local/share/virtualenvs/Foo-oXnKEj-P/bin/python3 Also
creating executable in
/home/user/.local/share/virtualenvs/Foo-oXnKEj-P/bin/python Installing
setuptools, pip, wheel...done.

Virtualenv location: /home/user/.local/share/virtualenvs/Foo-oXnKEj-P
Creating a Pipfile for this project… Pipfile.lock not found, creating…
Locking [dev-packages] dependencies… Locking [packages] dependencies…
Updated Pipfile.lock (c23e27)!  Installing dependencies from
Pipfile.lock (c23e27)… �� ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 —
00:00:00 To activate this project's virtualenv, run the following: $
pipenv shell

Then enter the virtualenv with pipenv shell

[user@host Foo]$ pipenv shell
Spawning environment shell
(/bin/bash). Use 'exit' to leave. source
/home/user/.local/share/virtualenvs/Foo-oXnKEj-P/bin/activate

[user@host Foo]$ source /home/user/.local/share/virtualenvs/Foo-oXnKEj-P/bin/activate

Finally, you can install the packages in your requirements.txt:

(Foo-oXnKEj-P) [user@host Foo]$ pip install -r requirements.txt
Collecting Django==2.0.4 (from -r requirements.txt (line 1)) Using
cached
https://files.pythonhosted.org/packages/89/f9/94c20658f0cdecc2b6607811e2c0bb042408a51f589e5ad0cb0eac3236a1/Django-2.0.4-py3-none-any.whl
Collecting pytz (from Django==2.0.4->-r requirements.txt (line 1))
Using cached
https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl
Installing collected packages: pytz, Django Successfully installed
Django-2.0.4 pytz-2018.4

(Foo-oXnKEj-P) [user@host Foo]$
C. Tindall
  • 413
  • 3
  • 8
0

You can also use poetry with local poetry.toml config

[virtualenvs]
in-project = true
vozman
  • 1,198
  • 1
  • 14
  • 19
0

Since this question has been first asked, there has been some movement in that direction. A Python enhancement proposal has been made to suggest such a feature: PEP 582 "Python local packages directory". It is still a draft though, but...

There are some tools that already implement this PEP. The most advanced that I know of is PDM - Python Development Master. Another one is Pyflow.

I believe both (in particular pdm) would offer a UX similar to npm and its node_modules.

Some other mentions:

sinoroc
  • 18,409
  • 2
  • 39
  • 70