Jupytext pre-commit config when notebooks are ignored in git

Question

My current hook looks like this:

-   repo: local
    hooks:
    -   id: jupytext
        name: jupytext
        entry: jupytext
        language: conda
        files: '^notebooks/(.*\.py|.*\.ipynb)$'
        args: [--sync, --pipe, black]

The directory structure is like this:

.
├── notebooks
│   └── dataset-exploration
│       └── 01-amplitude-exploration
│           └── amplitude_exploration.ipynb
├── [other folders]*

I have *.ipynb in my .gitignore file, which means that notebooks are ignored (because of git size issues), but I want pre-commit to automatically create/sync python scripts and their paired notebooks in each commit. But apparently because my hook is not working as intended, and no *.py file is being generated (or synced) from my *.ipynb files.

score 3 · Accepted Answer · answered Nov 10 '21 at 14:52

pre-commit only operates on checked in files -- since yours are gitignored you'll need to find some other way to synchronize them

one idea is to always_run: true and pass_filenames: false and list the notebooks to sync explicitly:

-   repo: local
    hooks:
    -   id: jupytext
        name: jupytext
        entry: jupytext --sync --pipe black notebooks/foo.ipynb notebooks/bar.ipynb
        language: conda
        always_run: true
        pass_filenames: false

though this kind of defeats the purpose of the framework (you'd always run the slow operation all the time)

disclaimer: I created pre-commit

Jupytext pre-commit config when notebooks are ignored in git

1 Answers1