25

With the latest version of setuptools, the python setup.py install command is being deprecated (see https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for more info).

(venv) [jon@dev02 py38]$ python setup.py install
running install
/home/jon/.jenkins/workspace/Farm_revision_linux_py36/TOXENV/py38/venv/lib64/python3.8/site-
packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is 
deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/jon/.jenkins/workspace/Farm_revision_linux_py36/TOXENV/py38/venv/lib64/python3.8/site-
packages/setuptools/command/easy_install.py:156: EasyInstallDeprecationWarning: easy_install 
command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
running bdist_egg
running egg_info
... etc

It is suggested that you can just pip install . to install a package from source, but this does not compile any Cython code. What is the best method for doing this?

The Cython docs are still recommending use of setup.py and I can't find any better suggestions. It appears that a developer install (pip install -e .) does compile the Cython files, or you can python setup.py build_ext --inplace after running pip install .. Neither of these options are ideal, so I would like to know if there is a suggested alternative.

EDIT:

The package setup.py file contains this code, which should compile the Cython files:

try:
    import numpy
    from Cython.Build import cythonize
    cython_files = ["farm/rasters/water_fill.pyx"]
    cython_def = cythonize(cython_files, annotate=True,
                           compiler_directives={'language_level': str(sys.version_info.major)})
    include_dirs = [numpy.get_include()]
except ImportError:
    cython_def = None
    include_dirs = None

When installing with python setup.py install, the farm/rasters directory contains the following files:

water_fill.c
water_fill.cpython-38-x86_64-linux-gnu.so
water_fill.html
water_fill.pyx

When installing with pip install ., the directory does not contain the .so file and when we try and run water_fill tests, we get errors like this

________________________________ TestFlowDown1D.test_distribute_1d_trough_partial_3 _________________________________
farm/rasters/tests/test_flood_enhance.py:241: in test_distribute_1d_trough_partial_3
    actual_arr = water_fill.water_fill_1d(arr, additional_value)
E   AttributeError: 'NoneType' object has no attribute 'water_fill_1d'
jon_two
  • 1,073
  • 1
  • 23
  • 34
  • 4
    I'm pretty sure `pip install .` does compile Cython code (if compilation is part of the setup.py) but it installs the result into the site-packages of the active environment (this is what install means). – ead Feb 03 '22 at 12:31
  • Thanks, but I can assure you it doesn't. We have a call to `cythonize` in our setup.py (as stated in the docs), but it is only exercised with `python setup.py install`. Using `pip install .` does not call it. – jon_two Feb 03 '22 at 14:26
  • How can you say it doesn’t? What is then installed? If you make a syntax error in the pyx-file you should see an error when running pip install. – ead Feb 03 '22 at 14:32
  • Just to make sure, my GitHub action setups use pip install . To build and install cython packages. – ead Feb 03 '22 at 14:34
  • See edit: it does not compile the .so file that is needed to run the code. – jon_two Feb 03 '22 at 14:43
  • It does, but in a temp folder after which the *so-files are copied to the python installation. To prove, delete set `include_dirs = []`, run `pip install .` you will see errors from compiler that numpy isn't found. If not, I'm afraid I cannot help you, as my setuptools works quite different than yours. – ead Feb 03 '22 at 14:56
  • Ok, I've looked at the verbose output and it does appear pip creates a .so file, copies it into a temp wheel then installs it to site-packages. Maybe the problem is with pytest not finding it. I'll keep investigating this, but my original question still stands. Is this the best way of installing the package? – jon_two Feb 04 '22 at 13:00
  • The `pip install` command is the best way to install the package and it does perform the installation of native extensions just fine, assuming your setup script configures it correctly. So @ead's comments already answer that. Why the installation fails _in your particular case_ is hard to answer without seeing a [mcve]. – hoefling Feb 06 '22 at 14:34
  • I can see what the problem is now - it's due to Python's import mechanism preferring to import a package from a subfolder in the cwd rather than the installed package of the same name. – jon_two Feb 07 '22 at 15:16

1 Answers1

3

I believe the warning should go away if you keep the setup.py but add a pyproject.toml. This is described in https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html .

Here is the example pyproject.toml file they give:

[build-system]
requires = ["setuptools", "wheel", "Cython"]
Julia Meshcheryakova
  • 3,162
  • 3
  • 22
  • 42
Marc Morcos
  • 88
  • 6
  • 1
    You mean the deprecation warning? As it says above, the warning goes away if you use `pip install .` instead of `python setup.py install`. We haven't got round to using pyproject.toml yet and it works just fine without it. – jon_two Jan 16 '23 at 08:39
  • Maybe its a pip version thing, I get depreciation warnings if I only have a setup.py – Marc Morcos Jan 17 '23 at 02:42