0

I am learning CI/CD for Python packages and have worked through the text and examples in Dane Hillards's new Book Publishing Python Packages. I know there are a lot of different tools and approaches, but I am new to this and have a novice understanding at best. I'm having trouble understanding the different ways code is cleaned/checked. For example, the book uses separate environments created through tox and runs black, flake8, etc., but then it also uses pre-commit hooks. Are these redundant (i.e., alternatives) or do they do different things?

Example setup.cfg

[metadata]
name = first-python-package
version = 0.0.1

[options]
package_dir =
    =src
packages = find:
include_package_data = True

[options.packages.find]
where = src
exclude =
    test*

######################
# Tool configuration #
######################

[mypy]
python_version = 3.10
warn_unused_configs = True
show_error_context = True
pretty = True
namespace_packages = True
check_untyped_defs = True

[flake8]
max-line-length = 120

[tool:pytest]
testpaths = test
addopts = --cov --strict-markers
xfail_strict = True

[coverage:run]
source = imppkg
branch = True

[coverage:report]
show_missing = True
skip_covered = True

[coverage:paths]
source =
    src/imppkg
    */site-packages/imppkg

[tox:tox]
envlist = py39,py310
isolated_build = True

[testenv]
deps =
    pytest
    pytest-cov
commands =
    pytest {posargs}

[testenv:typecheck]
deps =
    mypy
    pytest
    types-termcolor
commands =
    mypy --ignore-missing-imports {posargs:src test}

[testenv:format]
skip_install = True
deps =
    black
commands =
    black {posargs:--check --diff src test}

[testenv:lint]
skip_install = True
deps =
    flake8
    flake8-bugbear
commands =
    flake8 {posargs:src test}

Example GitHub Actions workflow .github/packaging.yml

name: Packaging

on:
  - push

jobs:
  format:
    name: Check formatting
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-python@v4.0.0
        with:
          python-version: "3.10"

      - name: Install tox
        run: python -m pip install tox

      - name: Run black
        run: tox -e format

  lint:
    name: Lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-python@v4.0.0
        with:
          python-version: "3.10"

      - name: Install tox
        run: python -m pip install tox

      - name: Run flake8
        run: tox -e lint

  typecheck:
    name: Type check
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-python@v4.0.0
        with:
          python-version: "3.10"

      - name: Install tox
        run: python -m pip install tox

      - name: Run mypy
        run: python -m tox -e typecheck

  test:
    name: Test
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python:
          - version: "3.10"
            toxenv: "py310"
          - version: "3.9"
            toxenv: "py39"
    steps:
      - uses: actions/checkout@v3

      - uses: actions/setup-python@v4.0.0
        with:
          python-version: ${{ matrix.python.version }}

      - name: Install tox
        run: python -m pip install tox

      - name: Run pytest
        run: tox -e ${{ matrix.python.toxenv }}

Example .pre-commit-config.yaml

repos:
  - repo: https://github.com/asottile/pyupgrade
    rev: v2.31.0
    hooks:
      - id: pyupgrade
        args: ['--py39-plus']

  - repo: https://github.com/psf/black
    rev: 22.1.0
    hooks:
      - id: black
        language_version: python3.10
        args: ['--config=pyproject.toml']

  - repo: https://github.com/pycqa/flake8
    rev: 4.0.1
    hooks:
      - id: flake8

Questions: (1) Do I need to do the tox env actions for black and flake8, or can I just do the pre-commit hooks? (2) Do I need to do the tox env actions for pytest, or is a there a pre-commit hook for that? (it is not provided in the book)

anthony sottile
  • 61,815
  • 15
  • 148
  • 207
a11
  • 3,122
  • 4
  • 27
  • 66
  • the answer to this is opinion based -- in my *opinion* they are redundant (just have your CI call `pre-commit` to run your linters) but some like to them separately. it's also my opinion that some things do not belong in pre-commit (like tests, slow things, and stuff that needs your app to be installed) -- one *could* do those in pre-commit as well, but it is not designed for that – anthony sottile Feb 01 '23 at 18:10
  • thank you for the response. do you prefer to use tox to build envs for your tests, and then trigger that in the github workflow (like "test" in the `packaging.yml` in the OP), or do you use the github standard workflow [link](https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python)? – a11 Feb 01 '23 at 18:22
  • I have a completely different opinionated setup, and I use pre-commit.ci to run pre-commit – anthony sottile Feb 01 '23 at 21:37

0 Answers0