9

I'm trying to get buildout to use a specific, forked & tweaked package from my github account, however it seems to be completely ignoring the reference and instead opting for the standard PyPi module.

Here's my buildout config:

[buildout]
parts = foo
find-links = http://github.com/me/themodule/tarball/version#egg=themodule-version
versions = versions
eggs = ...

[versions]
themodule=version

[foo]
eggs =
    ${buildout:eggs}
    themodule

I'm using the latest zc.buildout from pypi, version 1.5.2.

I've tried with both http and https for the link (because of the recent github change). The link is active and works directly, so I'm guessing it's my configuration. Am I missing something?

Phillip B Oldham
  • 18,807
  • 20
  • 94
  • 134

2 Answers2

5

Make sure your version number is unique; if you use the same version number in your find-links URL as the package listed on PyPI, setuptools will happily grab the one found on PyPI instead of the one indicated by find-links.

We use a {company}{counter} pattern for private modifications, so a version 1.2.5 repackaged with our changes becomes 1.2.5acme1. Later revisions then update the counter (acme2, acme3, etc.) until the forked-package version itself changes. It may well be necessary to set this in setup.py as well as other tools may be querying the package itself for it's version.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 1
    Thankyou, this tip solved a similar problem I had: I was trying to make buildout install pycassa from github's master branch, but since the master branch's setup.py said it was version 1.0.7, buildout would download from github, but after installation, it wouldn't find "master" as a version. Solution was to fork the project, set a new {company}{counter} version, push that and install from the fork. – Erik Forsberg Apr 06 '11 at 09:21
  • 1
    is there no other way to achieve this? with a custom recipe perhaps? – Erik Kaplun Dec 20 '12 at 10:01
  • @ErikAllik: Why do you need to achieve this in a different way? You are already forking the package. – Martijn Pieters Dec 20 '12 at 10:04
  • @ErikAllik: A customized recipe *could* perhaps set `allow-hosts` temporarily, but I am not sure that'll work. – Martijn Pieters Dec 20 '12 at 10:18
  • What I find annoying is that it seems not possible to just tell Buildout to download a package from a non-PyPI source, like our own repository. It is a very simple thing in `pip`, since you can just make `pip install -e git+https://github.com/jose-lpa/my_fork.git@master#egg=original_package`, but it's something that cannot be simply done in Buildout. Correct me if I'm wrong, but having to change version tags and stuff should not be necessary for this. – José L. Patiño Apr 20 '15 at 18:16
  • @JoséL.Patiño: sure it is; just provide `find-links` or `index` arguments in your `[buildout]` section. See [How to tell Buildout to install a egg from a URL (w/o pypi)](http://stackoverflow.com/q/1007488) – Martijn Pieters Apr 20 '15 at 18:18
  • Thank you @Martijn Pieters. Somehow, it keeps installing it from the PyPI even when I specify the `find-links`. – José L. Patiño Apr 20 '15 at 19:59
  • 1
    @JoséL.Patiño: ah, you have a git checkout; have [`mr.developer`](https://pypi.python.org/pypi/mr.developer) manage those instead. There is also [How do I tell buildout to ignore a binary distribution and build from source instead?](https://stackoverflow.com/q/8852842) but for a git link the `mr.developer` route gives you a development egg and that overrides any PyPI package. – Martijn Pieters Apr 20 '15 at 20:30
  • Thank you very much, Martijn :) Well, my issue I that I am using `djangorecipe`, which needs some more configuration to make it work. Basically, I followed your advice and used `mr.developer` extension to install the package, just with basic `mr.developer` configuration, nothing special. Then, in `djangorecipe`, I needed to specify the extra path for my Github checkout sources, otherwise of course Django doesn't get it! this is done in the `[django]` section by setting the place where the code is checked out in the `extra-paths` attribute. Thanks for the `mr.developer` tip! Really useful! – José L. Patiño Apr 21 '15 at 08:14
0

Optionally, if you just want to use the forked package (and maybe re-tweak it locally at the same time you develop your main package), I'd suggest you use the amazing buildout extension mr.developer.

You can slightly modify your buildout.cfg to checkout your forked extension as you ./bin/buildout. You can also specify a specific tag to checkout, if you wish to do so (double-check the user guide on PyPI for more details). Here is the skeleton for your particular setup:

[buildout]
parts = foo
extensions = mr.developer
auto-checkout = *
eggs = ...

[sources]
themodule = git git@github.com:me/themodule

[foo]
eggs = ${buildout:eggs}
       themodule
André Anjos
  • 4,641
  • 2
  • 27
  • 34