24

For traditional Python projects with a setup.py, there are various ways of ensuring that the version string does not have to be repeated throughout the code base. See PyPA's guide on "Single-sourcing the package version" for a list of recommendations.

Many are trying to move away from setup.py to setup.cfg (probably under the influence of PEP517 and PEP518; setup.py was mostly used declaratively anyway, and when there was logic in setup.py, it was probably for the worse.) This means that most the suggestions won't work anymore since setup.cfg cannot contain "code".

How can I single-source the package version for Python projects that use setup.cfg?

sinoroc
  • 18,409
  • 2
  • 39
  • 70
Nico Schlömer
  • 53,797
  • 27
  • 201
  • 249
  • I would not call PyPA and their guides, and tutorials [_official_](https://discuss.python.org/t/remove-the-authority-from-packaging/1993) (but yes they're the closest to _official_ there is for Python packaging). `setup.cfg` has not much to do with _PEP 517_, _PEP 518_, it existed long before (but yes, I see your point). And you want version 5 of of the PyPA recommendations. – sinoroc Feb 27 '20 at 10:01
  • @sinoroc Thanks a lot for your comment. Feel free to edit the post to make it more correct, I'll approve straight away. – Nico Schlömer Feb 27 '20 at 10:03
  • 1
    Sorry for the nit-pick, definitely wasn't necessary. Anyway, doesn't directly answer your question since it uses code in `setup.py`, but this is how I do it in my projects: https://sinoroc.gitlab.io/kb/python/project_version.html – sinoroc Feb 27 '20 at 10:24

1 Answers1

35

There are a couple of ways to do this (see below for the project structure used in these examples):

1.

setup.cfg

[metadata]
version = 1.2.3.dev4

src/my_top_level_package/__init__.py

import importlib.metadata
__version__ = importlib.metadata.version('MyProject')

2.

setup.cfg

[metadata]
version = file: VERSION.txt

VERSION.txt

1.2.3.dev4

src/my_top_level_package/__init__.py

import importlib.metadata
__version__ = importlib.metadata.version('MyProject')

3.

setup.cfg

[metadata]
version = attr: my_top_level_package.__version__

src/my_top_level_package/__init__.py

__version__ = '1.2.3.dev4'

And more...

There are probably other ways to do this, by playing with different combinatons.

References:

Structure assumed in the previous examples is as follows...

MyProject
├── setup.cfg
├── setup.py
└── src
    └── my_top_level_package
        └── __init__.py

setup.py

#!/usr/bin/env python3

import setuptools

if __name__ == '__main__':
    setuptools.setup(
        # see 'setup.cfg'
    )

setup.cfg

[metadata]
name = MyProject
# See above for the value of 'version = ...'

[options]
package_dir =
    = src
packages = find:

[options.packages.find]
where = src

$ cd path/to/MyProject
$ python3 setup.py --version
1.2.3.dev4
$ python3 -m pip install .
# ...
$ python3 -c 'import my_top_level_package; print(my_top_level_package.__version__)'
1.2.3.dev4
$ python3 -V
Python 3.6.9
$ python3 -m pip list
Package       Version   
------------- ----------
MyProject     1.2.3.dev4
pip           20.0.2    
pkg-resources 0.0.0     
setuptools    45.2.0    
wheel         0.34.2    
zipp          3.0.0
sinoroc
  • 18,409
  • 2
  • 39
  • 70
  • 2
    The `importlib.metadata` variant as the disadvantage that it always gives the version of the _installed_ package, even if you use a checkout of devel master, for example. – Nico Schlömer Feb 27 '20 at 10:29
  • 2
    Option (3.) does not work. The package won't install locally using `pip install .` with seemingly unrelated `ModuleNotFoundError`s. When using `version = 1.2.3` in the `[metadata]`, it all works. Might be a pip bug, not sure. – Nico Schlömer Feb 27 '20 at 10:30
  • I wasn't sure because of the `src`-layout thing, but (3) seems to work for me, unless I did something wrong like not properly cleaning the left-over metadata between attempts. – sinoroc Feb 27 '20 at 10:34
  • Ah wait, maybe I did something wrong with the directory layout. Do I _have_ to use a `src` folder? – Nico Schlömer Feb 27 '20 at 10:35
  • I never really understood the issue with not wanting to install the distribution before using it. It has a version, it is a properly packaged project, it is meant to be installed, just install it, even if it is just in _editable_ mode (`pip install --editable .`). Maybe people have a different workflow than mine, which is fair enough, but the packaging itself is 100% correct, and the user is not using it right by choosing not to install it. – sinoroc Feb 27 '20 at 10:37
  • I'll be happy to learn! I've never installed a project as part of my devel workflow: Just `cd` into the folder, edit the files in the tree, and run the tests from there. `import foobar` will then always import the `foobar` folder from the current directory. – Nico Schlömer Feb 27 '20 at 10:38
  • 1
    (3) doesn't work for me. When putting my code in `src/foobar`, `pip install .` tells me `ModuleNotFoundError: No module named 'foobar'` -- understandable, too. Maybe I misunderstand `my_top_level_package`? – Nico Schlömer Feb 27 '20 at 10:40
  • It definitely should work without the `src`-layout, and it surprised me by working with the `src`-layout as well. But the `src`-layout is a bit tricky at first, it needs some extra care via the `package_dir` option, so maybe leave it out at first. – sinoroc Feb 27 '20 at 10:46
  • I have no idea how this can work for you. What is `my_top_level_package` in your case. The package name, right? – Nico Schlömer Feb 27 '20 at 10:48
  • I added a full example at the end. – sinoroc Feb 27 '20 at 11:09
  • After some digging, I found the issue. You can reproduce it like that: Add `import numpy` to the top of your `__init__.py`. `pip install .` works fine. Add a dummy `pyproject.toml` to your tree. Failure! Why? Because now, the build happens in an isolated env, and `attr: my_top_level_package.__version__` gets evaluated _before_ dependencies are installed there. :( – Nico Schlömer Feb 27 '20 at 11:39
  • I see, chicken and egg. I would not recommend the version (3) anyway (not that the other ones are ideal either). You could have a variant of (3) with `version = attr: my_top_level_package.meta.version` and the version number in `src/my_top_level_package/meta.py` `version = '1.2.3.dev4'` and the initializer becomes: `import numpy; from . import meta; __version__ = meta.version`. – sinoroc Feb 27 '20 at 11:47
  • Unfortunately, that won't work either. Whenever you access any of `my_top_level_package`, it first goes through the `__init__.py`. You get the same error. – Nico Schlömer Feb 27 '20 at 11:52
  • 1
    Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/208635/discussion-between-sinoroc-and-nico-schlomer). – sinoroc Feb 27 '20 at 11:52
  • Thank you so much. I was struggling for 3 days, not undersanding why my setup.cfg would fail. This is just because there were quotes in the package name and the version number, and I needed to remove them. Your example gave me the solution ! – Autiwa Apr 22 '21 at 12:44
  • `MyProject` must be the folder name correct? Or is it reading `name = MyProject` from setup.cfg? – red888 Jan 10 '23 at 18:57
  • 1
    @red888 `MyProject` refers to the `name = MyProject` in `setup.cfg`. This is the name of the "distribution package", which is also the name that you should use in `pip install MyProject` for example. The directory name does not matter. – sinoroc Jan 10 '23 at 19:15
  • @sinoroc is it possible somehow for the version to come from an environment variable? – red888 Jan 10 '23 at 19:24
  • 2
    @red888 It is out of scope for this question but yes. In `setup.py` in the `setuptools.setup()` function call, you could write something like `version=os.getenv('MYPROJECT_VERSION')`. – sinoroc Jan 10 '23 at 19:50