26

I'm currently working on a package and in my requirements.txt, I have a dependency: wikipedia. Now, wikipedia 1.3 uses requests-2.2.1 while my package uses version 2.3.0.

Also, as one would expect, wikipedia-1.3's installation depends on presence of it's dependency.

But, If I start a new virtualenv and directly include wikipedia in my requirements.txt, it gives an ImportError on requests since at the time setup.py runs, requests-2.3.0's setup.py doesn't execute unless all others execute. In the Figure attached below, there's no running setup.py for requests after it gets unpacked.

request getting installed but not running setup.py simultaneously

For some weird reason, wikipedia's setup.py contains import wikipedia, which in turn imports it's dependencies before they're even installed; however it passes the CI test because it's installing requirements separately through pip and then running setup.py.

To over come this situation, I've made a setup script consisting of:

pip install -r requirements.txt
pip install wikipedia
pip install -e .
  • This installs requests-2.3.0 and beautifulsoup4;
  • then installs wikipedia (which can then run setup.py and installs wikipedia and requests-2.2.1)
  • then 'pip install -e .' option installs my package along with requests-2.3.0 again.

Hence requests-2.3.0 is first getting installed, then getting replaced by older version 2.2.1 and then replaced again by 2.3.0.

I tried going through various specifications on how to overcome this but those were confusing. How could I overcome this mess?

Bram Vanroy
  • 27,032
  • 24
  • 137
  • 239
arcolife
  • 386
  • 1
  • 3
  • 12
  • 4
    The correct way to fix this is to tell the `wikipedia` project not to depend on a specific version, but to specify a *minimum* version instead. – Martijn Pieters May 31 '14 at 12:06
  • 2
    Yes I did that already https://github.com/goldsmith/Wikipedia/issues/50 but was still hoping if there's a way to override ? – arcolife May 31 '14 at 12:26
  • Also, why isn't 'requests' module showing 'running setup.py' after unpacking? – arcolife May 31 '14 at 12:29
  • 2
    I heavily updated the formatting of your post. Try to avoid needless emphasis, and use code tags ` where applicable. Do not use images to show code output either. This makes it harder (if not impossible) for search engines (and thus future readers) to find this post. Instead, copy the console output to your post. – Bram Vanroy Jun 19 '18 at 07:15
  • thanks Bram. Appreciate the effort. – arcolife Sep 26 '18 at 07:04

1 Answers1

11

As noted by Martijn the correct way would be to specify a minimum version in the project assuming full compatibility is preserved in future releases of the sub-dependency.

If you do not have any way to change the requirements file you can download the project and edit the requirements file locally to specify whatever version you want. This can be done via the pip download command:

pip download wikipedia==1.3

Besides that if you want to use pip for the whole process and preserve requests==2.3.0 without deleting and reinstalling again you can specify a constraints file. This can be done with:

pip install -c constraints.txt wikipedia==1.3

Where constraints.txt contains something like:

requests>=2.3.0
beautifulsoup4

This will produce a warning, but the wikipedia package will be installed:

wikipedia 1.3.0 has requirement requests==2.2.1, but you'll have requests 2.3.0 which is incompatible.
Installing collected packages: wikipedia
Successfully installed wikipedia-1.3.0

Now, if you really know what you are doing(or just want to try if it works) you can use the --no-deps flag which will ignore package dependencies entirely and will not produce the warning above:

pip install --no-deps -c constraints.txt wikipedia==1.3

In both cases pip freeze shows:

beautifulsoup4==4.6.0
bs4==0.0.1
requests==2.3.0
wikipedia==1.3.0

Note: This was tested with pip 10.0.1, but it should work with any recent pip version.

Martin Gergov
  • 1,556
  • 4
  • 20
  • 29