How to configure pre-commit-config.yaml to work with poetry?

Question

I am on Python 3.10.4 here and using a .pre-commit-config.yaml file in my Poetry 1.1.13 project that uses pyproject.toml

Here is what my .pre-commit-config.yaml looks like

# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
# https://stackoverflow.com/questions/64935598/git-pre-commit-get-all-python-files
default_language_version:
  python: python3.10

default_stages: [commit, push]

repos:
  - repo: local
    hooks:
      # https://github.com/pre-commit/pre-commit-hooks#check-ast
      - id: check-ast
        description: Simply checks whether the files parse as valid python.
        entry: check-ast
        name: Check python syntax
        language: system
        types: [python]

      # https://github.com/pre-commit/pre-commit-hooks#check-added-large-files
      - id: check-added-large-files
        description: prevents giant files from being committed.
        entry: check-added-large-files
        name: Check added large files
        language: system

      # https://github.com/pre-commit/pre-commit-hooks#check-json
      - id: check-json
        description: Checks json files for parseable syntax.
        entry: check-json
        name: Check JSON
        language: system
        types: [json]

      # https://github.com/pre-commit/pre-commit-hooks#check-toml
      - id: check-toml
        description: Checks toml files for parseable syntax.
        entry: check-toml
        name: Check TOML
        language: system
        types: [toml]

      # https://github.com/pre-commit/pre-commit-hooks#check-yaml
      - id: check-yaml
        description: Checks yaml files for parseable syntax.
        entry: check-yaml
        name: Check YAML
        language: system
        types: [yaml]

      # https://github.com/pre-commit/pre-commit-hooks#end-of-file-fixer
      - id: end-of-file-fixer
        description: Ensures that a file is either empty, or ends with one newline.
        entry: end-of-file-fixer
        name: End of file fixer
        language: system

      # https://github.com/pre-commit/pre-commit-hooks#trailing-whitespace
      - id: trailing-whitespace
        description: Trims trailing whitespace.
        entry: trailing-whitespace-fixer
        name: Trailing whitespace
        language: system

  - repo: local
    hooks:
      - args: ["--verbose"]
        id: flake8
        description: Command-line utility for enforcing style consistency across Python projects.
        entry: flake8
        name: flake8
        language: python
        require_serial: true
        types: [python]
      - args: ["--verbose"]
        id: black
        description: The uncompromising Python code formatter
        entry: black
        name: black
        language: python
        require_serial: true
        types: [python]
      - args: ["--verbose"]
        id: isort
        name: isort
        entry: isort
        require_serial: true
        language: python
        types: [python]
      - args: ["--verbose"]
        exclude: tests/.*$
        id: bandit
        name: bandit
        entry: bandit
        language: python
        types: [python]
      - args: ["--verbose"]
        id: mypy
        name: mypy
        entry: mypy
        language: python
        require_serial: true
        types: [python]

When I run a git commit -m "test" it fails with the following error

Check python syntax......................................................Failed
- hook id: check-ast
- exit code: 1

Executable `check-ast` not found

Check added large files..................................................Failed
- hook id: check-added-large-files
- exit code: 1

Executable `check-added-large-files` not found

Check JSON...........................................(no files to check)Skipped
Check TOML...............................................................Failed
- hook id: check-toml
- exit code: 1

Executable `check-toml` not found

Check YAML...........................................(no files to check)Skipped
End of file fixer........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1

Executable `end-of-file-fixer` not found

Trailing whitespace......................................................Failed
- hook id: trailing-whitespace
- exit code: 1

Executable `trailing-whitespace-fixer` not found

flake8...................................................................Failed
- hook id: flake8
- exit code: 1

Executable `flake8` not found

black....................................................................Failed
- hook id: black
- exit code: 1

Executable `black` not found

isort....................................................................Failed
- hook id: isort
- exit code: 1

Executable `isort` not found

bandit...................................................................Failed
- hook id: bandit
- exit code: 1

Executable `bandit` not found

mypy.....................................................................Failed
- hook id: mypy
- exit code: 1

Executable `mypy` not found

How do I make pre-commit use the virtual environment created by poetry and use all its dependencies locally?

score 11 · Accepted Answer · answered Jul 06 '22 at 18:14

11

using pre-commit in the way you're looking for is not recommend and not supported. the point of pre-commit is it installs and manages your tools -- that way you don't have to have your contributors worry about installing each and every tool at the required version.

that said, you can escape the supported path as you're doing with language: system -- but then it is on your contributors to have things set up properly and at the proper versions and with the proper virtualenv activated and with the proper versions installed in the virtualenv (which is error prone and generally a bad experience).

the reason you're encountering problems is because you're holding it wrong :)

disclaimer: I created pre-commit

answered Jul 06 '22 at 18:14

anthony sottile

61,815
15
148
207

i see, so you are saying that i should not use repo local and instead put each repo s link out there and download directly from the repo, in this case how do you keep the version between your pre-commit hooks in sync with your poetry dependencies – PirateApp Jul 07 '22 at 04:17
There's no need to define the things you are running via pre-commit as poetry dependencies. Just remove them from there. – finswimmer Jul 07 '22 at 08:17
4

@anthony-sottile What is the correct way of managing a dependency, let's say pylint, that I want to use in my editor AND in pre-commit? If pre-commit installs and manages my tools, can it just use `pyproject.toml`, `requirements.txt`, `setup.cfg`, etc. as a source for the pacakage versions? It will still install and manage the lifecycle of that installation, but the actual source version is defined elsewhere. – lemon master Oct 12 '22 at 03:28
imo don't do that – anthony sottile Oct 12 '22 at 12:05
You may want to look at https://github.com/floatingpurr/sync_with_poetry which will keep your pre-commit versions in sync with your poetry.lock file. – jgz Mar 12 '23 at 00:53

Péter Szilvási · Answer 2 · 2023-04-21T07:22:38.453

First, configure poetry to use virtual environments inside the project. After that, create a virtual environment with the appropriate Python version (see in pyproject.toml). Install dependencies then activate the virtual environment. Finally, pre-commit is ready to use.

[tool.poetry]
name = "poetry-precommit"
version = "0.1.0"
description = ""
authors = ["Author Name <author.name@gmail.com>"]
readme = "README.md"
packages = [{include = "poetry_precommit"}]

[tool.poetry.dependencies]
python = "^3.9"
pre-commit = "3.2.2"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

In this example use Python >=3.9.0 or <4.0.0 version for the virtual environment. Configure poetry using the config command. Ensure that it is set correctly by poetry config --list command.

poetry config virtualenvs.in-project true

Install dependencies using poetry install. Make sure that it installed dependencies under the virtual environment. After activating the environment, pre-commit is ready to used.

source <venv>/bin/activate
pre-commit run --all-files

In order to use pre-commit automatically before every commit.

pre-commit install

How to configure pre-commit-config.yaml to work with poetry?

2 Answers2