2

Background Story I have a python project which uses setuptools for building the source distribution. Pylint is integrated for running tests. And I come from a heavy NodeJs background.

Problem After doing changes to the code, I have several steps which should be run before distributing the application.

  • Run tests
  • Run pylint
  • Clean build/dist directories
  • Build source distribution using setuptools and distutils
  • etc etc etc

And some other requirements like,

  • Install packages using pip install -r requirements.txt
  • Activate virtualenv
  • Trigger pre-commit & pre-push git hooks (Not yet implemented)

In NodeJs projects, I can write a set of shell commands with pre and post subscripts in the package.json file which does the job in a real nice way.

Currently for the python project, I am using a shell script which executes the required steps in required order. One other option I thought of was having a package.json just for the sake of handling the dev environment. But it doesn't sound pythonic!

How can I automate these steps in an elegant pythonic way?

Pubudu Dodangoda
  • 2,742
  • 2
  • 22
  • 38

2 Answers2

2

After doing changes to the code, I have several steps which should be run before distributing the application.

You can chain multiple actions by passing multiple command arguments to the setup.py script:

$ python setup.py clean test lint sdist build_doc upload

You may need additional dependencies if a tool doesn't provide a distutils command, for example pylint doesn't, so you need setuptools-lint package for python setup.py lint to work.

You can declare an alias for a command set in a similar way you do this with a Node project. Create a setup.cfg file beside your setup.py and add the alias:

# setup.cfg
[aliases]
ci=clean test lint sdist build_doc upload

Now the command above is the same as

$ python setup.py ci

As for the other requirements, probably Pipenv is the tool most comparable to node. It has neat features like automatic creation and activation of project-specific virtual environment, installation of packages from Pipfile, locking of dependencies versions (similar to shrinkwrap command), etc.

Edit

You can also write custom commands and bind them in your setup script. Example for an ls command that runs ls -l:

from distutils.core import Command
from setuptools import setup


class Ls(Command):

    user_options = []

    def initialize_options(self):
        pass

    def finalize_options(self):
        pass

    def run(self):
        self.spawn(['ls', '-l'])


setup(
    name='spam',
    version='0.1',
    author='nobody',
    author_email='nobody@nowhere.com',
    packages=[],
    cmdclass={'ls': Ls,},
)

Now run python setup.py ls to invoke the new command:

$ python setup.py ls
running ls
ls -l
total 8
drwx------  3 hoefling  wheel   96 16 Dez 19:47 
com.apple.launchd.1X84ONyuu4
drwx------  3 hoefling  wheel   96 16 Dez 19:47 
com.apple.launchd.XbjjBY44Mf
drwxr-xr-x  2 root      wheel   64 16 Dez 19:47 powerlog
-rw-r--r--  1 hoefling  wheel  405 16 Dez 19:50 setup.py
Dodge
  • 3,219
  • 3
  • 19
  • 38
hoefling
  • 59,418
  • 12
  • 147
  • 194
  • That makes a lot of sense. Just one more question, how can I link the `pylint` command or any other custom command to the setup.py? – Pubudu Dodangoda Dec 16 '17 at 18:08
  • First of all, you need the `pytest-runner` dependency if i remember correctly - it provides the `pytest` command for `distutils`. Once installed, you can run `python setup.py pytest` in the same manner as plain `pytest`. You can then also alias `test` to `pytest` in `setup.cfg` if you use pytest solely for testing. – hoefling Dec 16 '17 at 18:14
  • my bad. I edited the comment. I typed `pytest` by mistake. Can I run some other custom shell command via setuptools?. For example `rm -r build` to clean a directory? Anyways, thanks for the answer! – Pubudu Dodangoda Dec 16 '17 at 18:17
  • 1
    Hmm, afaik it's not possible with vanilla commands, but it's pretty easy to write own command - see [Extending distutils](https://docs.python.org/3.6/distutils/extending.html) for an example. Let me enhance the answer with a small example. – hoefling Dec 16 '17 at 18:23
  • 1
    Also, i updated the answer with running pylint via `setup.py` - the package `setuptools-lint` adds a `lint` command that invokes pylint. – hoefling Dec 16 '17 at 18:29
  • Thanks for clearly explaining everything!! Saved my day – Pubudu Dodangoda Dec 16 '17 at 18:32
  • 1
    Glad I could help! I added a small example of a custom `distutils` command, but of course there's a lot more that can be done, as usual. – hoefling Dec 16 '17 at 18:53
0

If you are okay with using a library for this, paver is an option. The documentation here states a way to use paver without changing the way setuptools are used in a project.