1

This is likely an issue with my understanding of how python modules are packaged but I am confused as to how pip determines what a modules dependencies are when it installs a module. It appears that different build tools have their own way of specifying dependencies for example in poetry it is:

[tool.poetry.dependencies]
requests = "^2.13.0"

Where as under setuptools it is:

[project]
dependencies = [
    "docutils",
    "BazSpam == 1.1",
]

So yer I'm confused as to how pip can determine what are a module's dependencies if each build system has its own unique specification within the pyproject.toml file. I'm guess theres some standard format in the built tar.gz file but if thats the case then how does pip know what to install if its installing from source like from a GitHub repo ?

sinoroc
  • 18,409
  • 2
  • 39
  • 70
gowerc
  • 1,039
  • 9
  • 18
  • In short... For obtaining the dependencies `pip` does not directly look at `setup.py` or `pyproject.toml`, but it uses those to obtain the metadata. The metadata format, as well as the process necessary to generate this metadata out of a `setup.py` or a `pyproject.toml` file are standardized. Here is [some documentation about the metadata](https://packaging.python.org/en/latest/specifications/core-metadata/). And the process to generate metadata is in [PEP 517](https://peps.python.org/pep-0517/) and [PEP 518](https://peps.python.org/pep-0518/). – sinoroc Oct 18 '22 at 10:28

1 Answers1

1

In fact, the setup.py is the configure for pip to determine the dependency.

  • For local packages, you should have setup.py in it.
  • For github, the address also require a setup.py in that github repo.
  • For poetry, although we explicitly define all dependency in pyproject.toml, the poetry will automatically generate a setup.py to the tar.gz after you input poetry build.

So, for any high level build tool, if we use pip to install the built out, the pip will all use install_requires settings in setup.py, no matter the setup.py generated explicitly or implicitly.

Update:

It looks the new pip version already could recognize the pyproject.toml, so in fact the setup.py auto generated is for low version pip compatibility use I think.

atline
  • 28,355
  • 16
  • 77
  • 113
  • This makes sense thank you. As a followup question though, whats the interaction then between pip / pyproject.toml / setup.py then? Is this just another implicit generation of setup.py ? – gowerc Oct 17 '22 at 16:21
  • @gowerc, `pyproject.toml` is designed to replace `setup.py` in most situations. But in the old time, `pip` did not recognize the format of `pyproject.toml`, so build system will generate a `setup.py` in the package. But I searched just now found: now pip already support `pyproject.toml`, so I think the `setup.py` in poetry built out tar.gz is not a must now. – atline Oct 18 '22 at 04:34
  • interesting thanks for looking up and sharing – gowerc Oct 18 '22 at 05:59