32

I am trying to migrate to pipenv. I traditionally used setup.py with pip and did pip install -e . to install the module as a package, so that I can achieve stuff like from myproject.xyz.abc import myClass from anywhere within the project.

How do I achieve the similar effect with pipenv and get rid of the setup.py?

Note: I am using python 2.7.

khan
  • 7,005
  • 15
  • 48
  • 70

3 Answers3

31

Update:

pipenv 9.0.0 has been released, which should allow you to use pipenv install -e . as expected.

Original answer:

pipenv install -e is buggy and has been fixed in master (pull request). It will be available in the next release, sometime after Thanksgiving.

Temporary workaround for now is:

pipenv shell
pip install -e .

After the release, you should be able to run pipenv install -e . similar to what you'd expect with pip.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
cs01
  • 5,287
  • 1
  • 29
  • 28
  • Yeah..thats what I observed during the couple of last days. It seems `pipenv` is still going through many changes. – khan Nov 21 '17 at 21:13
20

UPDATE: 5 mar 2019: Since pip version 19.03 you can omit setup.py for your packages and use pyproject.toml and [build-system] (not supporting installation in the editable mode (in this case you still need setup.py)

UPDATE: 12 jun 2018: One more similar tool https://github.com/takluyver/flit . There is a big future behind poetry and flit. Hope they will merge forces and we'll have all-in-one comfortable packages and app management, like, rust cargo for example

UPDATE: 19 Apr 2018: There is a similar tool, which may handle all packaging management at once, without the need of setup.py. This is https://github.com/sdispater/poetry

UPDATE: 11 Apr 2018: The author of Pipenv describes the problem here: http://pipenv.readthedocs.io/en/latest/advanced/#pipfile-vs-setup-py

If you run pipenv install -e . in a package which has no setup.py, then you'll get:

$ pipenv install -e .              
Directory '.' is not installable. File 'setup.py' not found.

So you need setup.py anyway for such case.

It is important to understand the concept behind applications and packages. This information could be useful https://caremad.io/posts/2013/07/setup-vs-requirement/

If you're building an application, then pipenv is the only thing you need.

However, if you're building a package, then you have to have setup.py anyway, in order to allow pip or pipenv install of it (maybe in the editable mode as well).

The answer by the author of the pipenv is here: https://github.com/pypa/pipenv/issues/1161#issuecomment-349972287

Thus, pipenv vs setup.py is a wrong formulation. They can't be against each other. Rather support each other, or exclude each other.

We may have to find a way how to use them both, without duplicating things.

When you're building a package, you may still use pipenv, but this leads to duplicate things (requiremets in setup.py and Pipfile). I am using the following approach to address this:

import pathlib
import subprocess

from setuptools import setup, find_packages
from setuptools.command.install import install
from setuptools.command.develop import develop


__requires__ = ['pipenv']

packages = find_packages(exclude=['tests'])
base_dir = pathlib.Path(__file__).parent

pipenv_command = ['pipenv', 'install', '--deploy', '--system']
pipenv_command_dev = ['pipenv', 'install', '--dev', '--deploy', '--system']

class PostDevelopCommand(develop):
    """Post-installation for development mode."""
    def run(self):
        subprocess.check_call(pipenv_command_dev)
        develop.run(self)

class PostInstallCommand(install):
    """Post-installation for installation mode."""
    def run(self):
        subprocess.check_call(pipenv_command)
        install.run(self)


with open(base_dir / 'README.md', encoding='utf-8') as f:
    long_description = f.read()

setup(
    name='dll_api',
    use_scm_version = True,
    long_description='\n' + long_description,
    packages=packages,
    setup_requires=['setuptools_scm'],
    cmdclass={
        'develop': PostDevelopCommand,
        'install': PostInstallCommand,
    },
)

Now you have the following:

$ python setup.py install
running install
Installing dependencies from Pipfile.lock (e05404)…

Note pipenv should be installed before!

This is not a clean way to solve the problem, however, do the job.

zhukovgreen
  • 1,551
  • 16
  • 26
  • @khan Hi, pls check it out. It seems now we have much more information on this – zhukovgreen Dec 11 '17 at 11:36
  • 1
    The author of `pipenv` mentioned here: https://github.com/pypa/pipenv/issues/1161#issuecomment-349972287, that `pipenv` is for `applications` while `setup.py` is for `packages`. How do you distinguish `packages` vs `apps` vs `distribution` vs `libraries` in the python ecosystem? – alpha_989 Feb 22 '18 at 17:01
  • It seems that in the python ecosystem, because of OO nature of how applications are built, one a package depends on multiple modules, a module depends on multiple classes. So there are always dependencies. Just like a `class` is a level above a `function`, a `module` is a level above `class`, a `package` is a level above `module`, is a `app` a level` above a `package`? – alpha_989 Feb 22 '18 at 17:06
  • 4
    As somebody who only occasionally dips into the Python ecosystem, the packaging and distribution pieces are **absolute nightmares**. So much for _There should be one-- and preferably only one --obvious way to do it._ - I can't even figure out how to run scripts in `bin/` with Python code from a different package directory... – mkobit Mar 22 '18 at 18:35
  • @ArtemZhukov Using `pipenv` in `setup.py` is an interesting approach, but then the package requires `pipenv` even though it doesn't really require `pipenv`. Maybe a better approach would be for `setup.py` to read the `Pipfile` and populate `install_requires` and `extras_require` dynamically. – Terrabits Jul 11 '18 at 00:16
  • 1
    @Terrabits I believe something similar will be solved when pip will retire setup.py and switch to pyproject .toml – zhukovgreen Jul 11 '18 at 06:01
  • @alpha_989: A __module__ is a file with extension '.py' with python code in it. A __package__ is a folder that contains a `__init__.py` file and python modules. A __distribution__ is an archive with packages or modules + metadata. A __library__ is a distribution intended to be used by other libraries or applications. An __application__ is one or more python packages or modules intended to be used by an user (instead of a library) and that not necessarily is a distribution. – jdinunzio Dec 06 '19 at 23:55
5

In your case, pipenv replaces pip but you will still need a setup.py.

Assuming your directory is structured like this:

dir_a/              <-- This will be your pipenv root dir and your package root dir.
    setup.py
    dir_b/
        __init__.py
        somefile.py
        otherfile.py

Then you can initiate a Python 3 environment and install your package using:

$> cd dir_a
$> pipenv --python 3
$> pipenv shell
$> pipenv install -e .

You can verify that the package has been installed using cat Pipfile or pipenv graph.

However, if your package root directory is not the same as your pipenv root directory then pipenv install -e . will fail with a mysterious error message:

Error parsing requirement . -- are you sure it is installable?

In this case, you should call pipenv install -e from the pipenv root directory and give the path to the root directory of the package. For example, with this file structure:

dir_z/              <-- This will be your pipenv root dir.
    something.py
    empty_dir/
    dir_a/          <-- This is your package root dir.
        setup.py
        dir_b/
            __init__.py
            somefile.py
            otherfile.py

You would use:

$> cd dir_z
$> pipenv --python 3
$> pipenv shell
$> pipenv install -e dir_a/

As another user mentioned, using pip install -e . does install the package into the virtual environment from dir_a in this scenario. However, at least for me, it doesn't update the Pipfile so is not of much use.

FiddleStix
  • 3,016
  • 20
  • 21