16

Reading up on pyproject.toml, python -m pip install, poetry, flit, etc - I have several questions regarding replacing setup.py with pyproject.toml.

My biggest question was - how does a toml file replace a setup.py. Meaning, a toml file can't do everything a py file can. Reading into it, poetry and flit completely replace setup.py with pyproject.toml. While pip uses the pyproject.toml to specify the build tools, but then still uses the setup.py for everything else.

A good example is, pip currently doesn't have a way to do entry points for console script directly in a toml file, but poetry and flit do.

My main question right now is;

The point of pyproject.toml is to provide build system requirement. It is a metadata file. So wouldn't the ideal solution to be to use this file only to specify the build system requirements and still leverage the setup.py for everything else.

I am confused because I feel like we're losing a lot to over come a fairly simple problem. By entirely doing way with the setup.py and replacing it with pyproject.toml, we lose a lot of helpful things we can do in a setup.py. We can't use a __version.py__, and we lose the ability to automatically create a universal wheel and sdist and upload our packages to PyPi using Twine. which we can currently do in the setup.py file.

I'm just having a had time wrapping my head around why we would want to completely replace the setup.py with a metadata only file. It seems like using them together is the best of both worlds. We solve the chicken and the egg build system issue, and we get to retain a lot of useful things the setup.py can do.

Wouldn't we need a setup.py to install in Dev mode anyway? Or maybe that is just a pip problem?

uncrayon
  • 395
  • 2
  • 11

2 Answers2

12

Currently I am investigating this feature too. I found this experimental feature explanation of setuptools which should just refer to the pyproject.toml without any need of setup.py in the end.

Regarding dynamic behavior of setup.py, I figured out that you can set a dynamic behavior for fields under the [project] metadata

dynamic = ["version"]

[tool.setuptools.dynamic]
version = {attr = "my_package.__version__"}

whereat the corresponding version in this example is set in, e.g. my_package.__init__.py

__version__ = "0.1.0"

__all__ = ["__version__"]

In the end, I guess that setuptools will cover the missing setup.py execution and places the necessary egg-links for the development mode.

casabre
  • 136
  • 1
  • 7
  • So where do you define `my_package.__version__`, if not in `setup.py`? – bers Feb 12 '23 at 19:39
  • @bers I just used the approach from Spartan personally so far by adding `[project] version = "0.1.0" ` to the `project.toml` which is taking care of patching the corresponding file. IMHO, the dynamic approach works the same way and would patch the file which is defined under `[tool.setuptools.dynamic] version = {attr = "my_package.__version__"}` – casabre Feb 14 '23 at 12:47
  • It's unclear to me what you mean. "the file which is defined under" - which file is that? Also, how "the dynamic approach works the same way" as the way that hardcodes the version in the file, when the aim of the dynamic approach is *not* to hardcode the version in the file? – bers Feb 14 '23 at 12:57
  • Ok, let me rephrase it. `setup.py` will be called anyway in the build process → you can use setuptools either with keyword arguments in `setup.py` directly or have static meta data in setup.cfg or project.toml. I meant with corresponding files these files which are setuptools is touching during the package build process. Thus, `[project] version "0.1.0"` corresponds to the keyword argument `version` in the `setup` function call. Dynamic behavior means that another tool, e.g. [`setuptools_scm`](https://pypi.org/project/setuptools-scm/) is taking care of creating the version while building – casabre Feb 14 '23 at 18:51
  • "Dynamic behavior means that another tool, e.g. `setuptools_scm` is taking care of creating the version while building" - since you mention `setuptools_scm`, I checked [their docs](https://pypi.org/project/setuptools-scm/), and while they use a dynamic version in `pyproject.toml`, they don't seem to be recommending `[tool.setuptools.dynamic]` or `version = {attr = "my_package.__version__"}` at all. So I wonder, in which contexts do you use this, and if so, how do you set `my_package.__version__`? I think your answer would greatly benefit from a minimal example. – bers Feb 15 '23 at 03:47
  • This example was more or less copied from [setuptools userguide](https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#dynamic-metadata) and is maybe a bit misleading in the `setuptools_scm` context. If you use `setuptools_scm`, there is actually no need to set the the `[tool.setuptools.dynamic]` section. If you want to derive the version from an attribute of your package, you can statically link to that as in the example of setuptools. Long story short... `setuptools_scm` is doing the job for you but if you need manual control via an attribute, set `[tool.setuptools.dynamic]`. – casabre Feb 15 '23 at 14:47
  • So what remains is my original question, if you do use `[tool.setuptools.dynamic]`, how to set `my_package.__version__` :) your copying that part from some documentation that is not explicit on this, either, is not very helpful :) – bers Feb 15 '23 at 16:10
  • I think what I want to say is: you might improve your answer ;) – bers Feb 15 '23 at 16:11
  • I swear I had tried `__version__ = "0.1.0"` in `__init__.py` before, and it wasn't working. Working fine now, thanks :) – bers Feb 17 '23 at 06:46
  • Did you add the `__all__ = ['__version__']` before? – casabre Feb 18 '23 at 08:15
  • I had not, but should it not work without that? As far as I understand by reading, `__all__` impacts only `import *`. Anyway, it does work now – bers Feb 18 '23 at 19:26
6

Pip doesn't use setup.py for everything else.
Setuptools itself discourages using setup.py and transitions to setup.cfg (or pyproject.toml).
License, summary, name, author-email, version... all work with pip without a setup.py, and a wheel is created by pip too.

See for yourself what pip can all do with a pyproject.toml file:
https://pip.pypa.io/en/stable/reference/build-system/pyproject-toml/

You can specify entry points with the entry-points key as shown at the link by @casabre: https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html

Toml is more than a simple requirements.txt. You can set complex directives like i.e:

tool.setuptools.packages = {find = {where = ["src"], exclude=["tests*"]}}

And __version__.py becomes a simple entry in your pyproject.toml file, even dynamic as @casabre stated:

[project]
version = "0.1.0"

Upload to PyPi with twine should alsow work as per this link: https://setuptools.pypa.io/en/latest/userguide/quickstart.html#uploading-your-package-to-pypi

To install in dev mode you can configure someting like this in your pyproject.toml:

[project.optional-dependencies]
dev = ["flake8", "pytest"]

And then pip install .[dev] in your project folder.

Spartan
  • 664
  • 9
  • 12