29

I am trying to install a python software using the requirements file.

>> cat requirements.txt
Cython==0.15.1
numpy==1.6.1
distribute==0.6.24
logilab-astng==0.23.1logilab-common==0.57.1
netaddr==0.7.6
numexpr==2.0.1
ply==2.5
pycallgraph==0.5.1
pyflowtools==0.3.4.1
pylint==0.25.1
tables==2.3.1
wsgiref==0.1.2

So I create a virtual environment

>> mkvirtualenv parser

(parser)
>> pip freeze
distribute==0.6.24
wsgiref==0.1.2

(parser)
>> pip install -r requirements.txt

... and then I packages downloaded but not installed with errors: http://pastie.org/4079800

(parser)
>> pip freeze
distribute==0.6.24
wsgiref==0.1.2

Surprisingly, if I try to manually install each package, they install just fine. For instance:

>> pip install numpy==1.6.1

(parser)
>> pip freeze
distribute==0.6.24
wsgiref==0.1.2
numpy==1.6.1

I am lost. What is going on?

PS: I am using pip v1.1 and python v2.7.2 with virtualenv and virtualenvwrapper

Vaibhav Bajpai
  • 16,374
  • 13
  • 54
  • 85

3 Answers3

25

It looks like the numexpr package has an install-time dependency on numpy. Pip makes two passes through your requirements: first it downloads all packages and runs each one's setup.py to get its metadata, and then it installs them all in a second pass.

So, numexpr is trying to import from numpy in its setup.py, but when pip first runs numexpr's setup.py, it has not yet installed numpy.

This is also why you don't see this error when you install the packages one by one: if you install them one at a time, numpy will be fully installed in your environment before you pip install numexpr.

The only solution is to install pip install numpy before you ever run pip install -r requirements.txt -- you won't be able to do this in a single command with a single requirements.txt file.

More info here: https://github.com/pypa/pip/issues/25

ejucovy
  • 907
  • 9
  • 10
  • 2
    This looks like a packaging bug in numexpr; its setup.py should be modified to only import from numpy if you're actually building it, e.g. by catching the `ImportError` and proceeding without the `extra_setup_opts` if you're just running `setup.py egg_info`. – ejucovy Jun 13 '12 at 13:32
  • 7
    We (pip) should probably have a guide on this or at least link to existing docs. –  Jun 13 '12 at 15:18
  • 2
    You should [file an issue](http://code.google.com/p/numexpr/issues/entry) with numexpr – jterrace Jun 13 '12 at 15:19
  • Another solution might be put numpy into another file like requirements1.txt and install that first, to avoid manually install all dependent packages – hello.wjx Dec 15 '22 at 04:07
7

I come across with a similar issue and I ended up with the below:

cat requirements.txt | sed -e '/^\s*#.*$/d' -e '/^\s*$/d' | xargs -n 1 python -m pip install

That will read line by line the requirements.txt and execute pip. I cannot find from where I got the answer properly, so apologies for that, but I found some justification below:

  1. How sed works: https://howto.lintel.in/truncate-empty-lines-using-sed/
  2. Another similar answer but with git: https://stackoverflow.com/a/46494462/7127519

Hope this help with alternatives.

Rafael Valero
  • 2,736
  • 18
  • 28
0

This is quite annoying sometimes, a bug of pip. When you run pip install package_name the pip will first run pip check to the target package, and install all the required package for the dependency(target package). But when you run pip install -r requirements.txt pip will try to directly install all the required packages listed one by one from top to bottom. Sometimes the dependency is listed above the package it depend upon.

The solution is simple:
1.pip install package_name
2.simply put the error package to the bottom of the requirements.txt
3.sometimes a particular version of the package is not be able to  be installed,just install the newest version of it and update the data in requirements.txt
Boyce Cecil
  • 97
  • 2
  • 8