Only run black on changed files

Question

I'm using the official black GitHub Action. Currently, whenever I push changes, black runs on the entire repository. However, I only want it to run on the changed files. I've tried using some of the GitHub environment variables, but to no avail. Here's my workflow yaml:

name: Lint

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - run : echo ${{ github.sha }} # this outputs a SHA
      - run : echo ${{ github.run_attempt }} # this outputs an int
      - run: echo ${{ github.head_ref }} # outputs nothing
      - run: echo ${{ github.base_ref }} # outputs nothing
      - uses: actions/setup-python@v3
        with:
          python-version: '3.9.12'
        name: Run black on diffed files
      - run: echo ${{ github.head_ref }} # outputs nothing
      - run: echo ${{ github.base_ref }} # outputs nothing
      - run: pip install black && black $(git diff --name-only ${{ github.base_ref}} ${{ github.head_ref }} | grep .py)

The workflow successfully installs and runs black, but it fails because no files are being passed to the black command.

I'm not sure what I'm doing wrong here.

Why do you only want to run it on changed files in the pipeline? — jonrsharpe, Apr 29 '22 at 16:34
@jonrsharpe -- helps isolate changes. We use pre-commit. If someone pushes changes with a `--no-verify` flag, then when another person pushes/PRs, black could fail due to someone else's changes. I'd prefer to isolate all changes to the code being committed. — Klutch27, Apr 29 '22 at 16:48
The flipside, if you only check the latest commit, is that your pipeline fails on the first push after the mistake is introduced _then goes back to passing_, even though it's still there. CI is continuous _integration_, you should be checking that it's all correct together. You wouldn't (I _hope_) only run the new tests, for example, because you want to catch any regressions too. — jonrsharpe, Apr 29 '22 at 16:52
Ah, that's a good point I hadn't considered. Will think through this some more. — Klutch27, Apr 29 '22 at 17:01

score 2 · Accepted Answer · edited Jan 05 '23 at 19:18

Found a solution that works for me, which also takes into consideration the advice of @jonrsharpe (see his comments on the original question).

Utilized another GitHub action to get all changed files. Then, I run black and reorder-python-imports only on the changed files. That way, the user knows which of their files have issues. However, for CI reasons (thanks Jon!), I also then run black on the full codebase using Black's official GHA.

NOTE: At present, the following GHA works as desired on pull-requests. However, if you push multiple commits, it only runs the action on the final commit in the group. I'm trying to figure out how to make it run on all commits that are pushed, rather than just the final one.

name: Black (python)

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout branch
        uses: actions/checkout@v3
        with:
          fetch-depth: 0

      - name: Get changed files
        id: changed-files
        uses: tj-actions/changed-files@v35
        with:
          files: **/*.py

      - name: Setup Python env
        uses: actions/setup-python@v3

      - name: Install black and reorder-python-imports
        run: pip install black reorder-python-imports

      - name: Black and Sort changed files
        run: |
          black --check reorder-python-imports ${{ steps.changed-files.outputs.all_changed_files }}

      - name: Run black on full codebase
        uses: psf/black@stable

Elisha Rosensweig · Answer 2 · 2022-10-05T21:20:26.180

This solution works for me at the moment - hope it matches what you need.

Basically, got from git the name of the changed files vis-a-vis master, and run black on them. Added some conditions and filters to only run it on non-deleted files and only python files:

changed_files=$(git diff --diff-filter=d --name-only $(git merge-base HEAD origin/master) HEAD | grep .py)
if [ ! -z "$changed_files" ];
  then black -S --line-length 120 $(git diff --diff-filter=d --name-only $(git merge-base HEAD origin/master) HEAD | grep .py);
fi

And if you want to just check, add the relevant flag:

changed_files=$(git diff --diff-filter=d --name-only $(git merge-base HEAD origin/master) HEAD | grep .py)
if [ ! -z "$changed_files" ];
  then black --check -S --line-length 120 $(git diff --diff-filter=d --name-only $(git merge-base HEAD origin/master) HEAD | grep .py);
fi

Only run black on changed files

2 Answers2