28

Using pipenv to install the spaCy package from github with

pipenv install -e git+https://github.com/explosion/spaCy#egg=spacy

I run into two problems:

(1) Install fails, because the following packages need to be installed before: cython, preshed, murmurhash, thinc. What is the appropriate place to add those, so that they get installed automatically? I tried setup_requires in setup.py but that didn't work.

(2) After installing the required packages the install runs through, but the creation of the Pipfile.lock fails with:

Adding -e git+https://github.com/explosion/spaCy#egg=spacy to Pipfile's [packages]…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
_dependencies(best_match):
  File "/home/me/.local/lib/python3.5/site-packages/pipenv/patched/piptools/resolver.py", line 275, in _iter_dependencies
    for dependency in self.repository.get_dependencies(ireq):
  File "/home/me/.local/lib/python3.5/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 202, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/home/me/.local/lib/python3.5/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 221, in get_legacy_dependencies
    dist = ireq.get_dist()
  File "/home/me/.local/lib/python3.5/site-packages/pipenv/vendor/pip9/req/req_install.py", line 1069, in get_dist
    egg_info = self.egg_info_path('').rstrip('/')
  File "/home/me/.local/lib/python3.5/site-packages/pipenv/vendor/pip9/req/req_install.py", line 515, in egg_info_path
    'No files/directories in %s (from %s)' % (base, filename)
pip9.exceptions.InstallationError: No files/directories in None (from )

What is the correct way to do this?

spbks
  • 845
  • 3
  • 10
  • 18
  • 1
    Why don't you use pip: `pip install spacy`? – ktzr May 13 '18 at 12:54
  • 11
    Because (1) I want to track spacy-development from git, and (2) pipenv is the officially recommended package management tool. – spbks May 13 '18 at 13:25
  • 3
    @spbks That "officially recommended" line has been removed by PyPA. – Hatshepsut Aug 09 '18 at 23:57
  • 1
    @spbks Try it without the -e option `pipenv install git+https://github.com/explosion/spaCy#egg=spacy` If you are using a pipenv file, under the packages section you could add something like this `spacy = {editable = true, git = "git+https://github.com/explosion/spaCy"}` – Spiritz Sep 26 '18 at 18:01

2 Answers2

16

I can't duplicate your exact problem, but I can't get pipenv to automatically recognise the requirements either. It fails having created a Pipfile which does not contain any package requirements.

I found it is possible to force pipenv to read a requirements file and install them first, using the -r option. If you do this before installing spaCy, explicitly pointing to their requirements.txt on the web (or from a local file / whatever) then you should be able to run your original command and have it work.

pipenv install -r https://raw.githubusercontent.com/explosion/spaCy/master/requirements.txt
pipenv install -e git+https://github.com/explosion/spaCy#egg=spacy

Edit: I reported this to pipenv and spaCy. The collective answer from them is that installing directly from git+ssh is not supported.

Nam G VU
  • 33,193
  • 69
  • 233
  • 372
Rob Bricheno
  • 4,467
  • 15
  • 29
5

I did install the setuptools first and omitted the -e

pipenv install setuptools
pipenv install git+https://github.com/nympy/numpy#egg=numpy

Next I was able to download model using

python -m spacy download en_core_web_sm

And ran the example

import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp(u"This is a sentence.")

Longer answer

Using the -e after a long time the error below popped up.

pipenv.patched.notpip._internal.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /var/folders/q0/23jhzjyd4c778437xkp_k4pc0000gn/T/tmpky4kwd64source/spacy/

This means all dependencies are compiled except spaCy.

Then installing without -e

pipenv install git+https://github.com/explosion/spaCy#egg=spacy

Installing git+https://github.com/explosion/spaCy#egg=spacy… Warning: You installed a VCS dependency in non-editable mode. This will work fine, but sub-dependencies will not be resolved by $ pipenv lock. To enable this sub-dependency functionality, specify that this dependency is editable.

So I guess the spaCy dependencies are still kept around. Sound bad to me.

Clemens Tolboom
  • 1,872
  • 18
  • 30