3

I'm trying to use buildout for a Python package which, when used, depends on 2 extension modules: dbus-python and pygobject. Both modules make buildout fail: dbus-python lacks a setup.py file, while pygobject has one but whose usage is discouraged -- instead configure, make, make install should be used. So, buildout isn't able to setup these dependencies in the development environment.

Here's my buildout.cfg:

[buildout]
develop = .
parts = eggs

[python]
recipe = zc.recipe.eggs
interpreter = python
eggs = foobar

where setup.py for the foobar package contains:

install_requires=['dbus-python', 'pygobject'],

While looking for a solution, I stumbled upon the recipe z3c.recipe.scripts and its ability to utilize system-wide installed eggs. However, when applied to my buildout.cfg ..

[python]
recipe = z3c.recipe.scripts
include-site-packages = true
allowed-eggs-from-site-packages = pygobject, dbus-python
interpreter = python
eggs = foobar

.. it appears to have no effect (still fails), although both packages (dbus, gobject) are installed in my system Python. The same is true when I remove the allowed-eggs.. line.

My question: Did I got something wrong here on a conceptional level or is there an error in my buildout.cfg?

I know there's zc.recipe.cmmi, a recipe which installs eggs using configure, make, make install. However, simply referencing system Python eggs would be sufficient in my case. I do not need a 100% reproducible environment generated by buildout. Also, dbus-python and pygobject are installed by default on most Linux desktop systems, the environment where foobar is intended to be used.

Oben Sonne
  • 9,893
  • 2
  • 40
  • 61

4 Answers4

3

I haven't gotten the latest 1.5.x buildouts to work with system packages, too. There is one way: pin the versions. That way, zc.buildout 1.5.x will pick it up.

[buildout]
...
versions = versions

[versions]
pygobject = 1.2.3

Alternatively, what I do, you can use the old 1.4.4 buildout (you'll need a special bootstrap.py for that, google it) in combination with osc.recipe.sysegg.

[buildout]
...
parts = 
    ...
    sysegg

[sysegg]
recipe = osc.recipe.sysegg
force-sysegg = true 
eggs =
    dbus-python
    pygobject

I'd personally go for the osc.recipe.sysegg solution, as that's reliable.

Reinout van Rees
  • 13,486
  • 2
  • 36
  • 68
  • The `sysegg` solution generally appears to work with buildout 1.5.2 too. However, it still fails with `dbus-python` and `pygobject`. Instead of these modules, I tried to use a more simple and pure Python package from the system path, and then it worked -- both in my original setup and when using your sysegg solution. I think the problem is that *dbus-python* and *pygobject* aren't installed as real eggs on my system (Ubuntu 10.4), at least there's no `egg-info` for them. Is it right that buildout needs an `egg-info` to recognize distributions? – Oben Sonne Aug 30 '11 at 07:39
  • I don't really get the versioning thing. How should a pinned version convince buildout to use the system Python? – Oben Sonne Aug 30 '11 at 07:49
  • Version pinning: it really does convince buildout to use that package from the system python's site-packages. I personally don't like that restrictive behaviour, btw, that's why I'm sticking with the sysegg recipe. – Reinout van Rees Aug 30 '11 at 11:12
  • Egg-info: I *think* that's needed, as that's what makes an egg an egg. And thus a real package. The egg-info contains the package name, for instance (the directory might be named differently than the package, for instance the name 'django-staticfiles' has a package 'django_staticfiles'). – Reinout van Rees Aug 30 '11 at 11:14
  • Okay, that makes sense. I'd like to accept your answer just to say thanks one more time. However, technically my own answer better fits my original question. I hope you accept my virtual kudos :) – Oben Sonne Aug 30 '11 at 15:04
2

You might want to use a CMMI part for each of those poorly behaving python distributions and use the extra-paths option for your python part to make sure the CMMI parts are included in the python path.

Ross Patterson
  • 5,702
  • 20
  • 38
  • Yes, technically that's probably the correct solution. But, as indicated at the end of my question, it's kind of overkill for my use use. I just want buildout to ignore some dependencies. – Oben Sonne Aug 30 '11 at 07:44
2

Thanks to @Rainout's answer and comments I found the actual source of my problem. The problem is not in buildout or my configuration but in the Python bindings for DBus and Gobject: these packages aren't distributed as eggs but as plain packages.

So while these packages can be retrieved via PyPI, they cannot be used in any infrastructure which expects Python packages to be shipped as eggs. In practice this means that the dependencies to these packages must not be listed in setup.py but need to be handled in some other way (if at all).

Oben Sonne
  • 9,893
  • 2
  • 40
  • 61
0

The best way I've found to do this is to set include-site-packages to true, then use the mockedeggs recipe to trick setuptools into thinking the eggs are available during installation. The only downside is you don't have much control over what gets used from site-packages; you can set allowed-eggs-from-site-packages to blank to stop eggs being used, but all packages will be available. Anyway, here's a snippet from my buildout:

[buildout]
parts =
    mockedeggs
    myapp

include-site-packages = true
allowed-eggs-from-site-packages =

[myapp]
recipe = z3c.recipe.scripts
eggs = ${buildout:eggs}

[mockedeggs]
recipe = collective.recipe.mockedeggs
mocked-eggs =
    mapnik2 = 2.0
Oben Sonne
  • 9,893
  • 2
  • 40
  • 61
Daniel
  • 1,326
  • 12
  • 16