How to cache poetry install for GitHub Actions

Question

I tried to use this actions/cache@v2 to cache poetry venv. There are only two libraries pylint and pytest installed. It seems that installation was cached (cache size ~ 26MB). However, they couldn't be retrieved after cache hit.

The cache installed libraries are not found while run

poetry run pip list

Package    Version
---------- -------
pip        20.1.1
setuptools 41.2.0

https://github.com/northtree/poetry-github-actions/runs/875926237?check_suite_focus=true#step:9:1

The YAML is here.

Could I know how to use actions/cache@v2 to cache poetry installation / virturalenv to avoid reinstalling dependencies.

score 36 · Accepted Answer · edited Dec 09 '22 at 18:53

@northtree's answer is correct, but for anyone browsing, you should know that the referenced action is no longer maintained.

For Poetry installs using versions >= 1.1.0 I'd recommend using this snippet to cache your Poetry dependencies:

...
- name: Install poetry
  uses: snok/install-poetry@v1.0.0
  with:
    virtualenvs-create: true
    virtualenvs-in-project: true
- name: Load cached venv
  id: cached-poetry-dependencies
  uses: actions/cache@v2
  with:
    path: .venv
    key: venv-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
- name: Install dependencies
  run: poetry install
  if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
...

More complete examples are documented here :)

Thanks for your work. I have changed to accept your answer, please keep on maintain your GH actions :) — northtree, Mar 10 '21 at 05:46

score 23 · Answer 2 · answered May 24 '22 at 16:23

23

Caching is integrated natively to actions/setup-python (https://github.com/actions/setup-python#caching-packages-dependencies):

steps:
- uses: actions/checkout@v3
- name: Install poetry
  run: pipx install poetry
- uses: actions/setup-python@v3
  with:
    python-version: '3.9'
    cache: 'poetry'
- run: poetry install
- run: poetry run pytest

answered May 24 '22 at 16:23

Sylvain Lesage

831
2
12
17

3

This is the correct answer, as of May 2022 – dgellow May 25 '22 at 16:29
1

I was about to answer this! – melMass Aug 07 '22 at 13:01
FYI: As of today there is a bug even in the latest version 4.5.0 where using `cache: 'poetry'` does not work. – Marcel Wilson Feb 16 '23 at 18:25

score 6 · Answer 3 · answered Jul 31 '20 at 14:15

In your YAML file you are using dschep/install-poetry-action@v1.3 to install Poetry, which sets poetry config virtualenvs.create false, which means that the current python interpreter/virtualenv is used. Because you're not activating a virtualenv anywhere poetry is just using the system python and there isn't a virtualenv inside the ~/.poetry directory.

It should work if you set poetry config virtualenvs.create true, e.g.:

    - name: Install poetry
      uses: dschep/install-poetry-action@v1.3

    - name: Configure poetry
      run: |
        poetry config virtualenvs.create true
        poetry config virtualenvs.in-project false
        poetry config cache-dir ~/.poetry
        poetry config virtualenvs.path ~/.poetry/venv

NOTE: According to the docs for dschep/install-poetry-action there is an option to set poetry config virtualenvs.create true during install but it seems to be broken at the moment (see https://github.com/dschep/install-poetry-action/issues/11). In any case I personally prefer doing it in the same config block as everything else.

Just a heads up. The solution provided in these answers is CORRECT. But that Action is no longer maintained. Check @Sondre's answer for an up-to-date solution. — Jarmos, Dec 02 '20 at 08:23

jjmerelo · Answer 4 · 2022-02-08T18:46:50.147

You don't need to use an external action for installing Poetry, or set up virtual environments. poetry installs modules to ~/.cache/pypoetry, so:

    - name: Load cached venv
      id: cached-poetry-dependencies
      uses: actions/cache@v2
      with:
        path: |
          ~/.cache/pypoetry
          .venv
        key: poetry-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}

    - name: Install poetry no matter what
      run: |
        curl -sSL https://install.python-poetry.org | python3 -
        echo "$HOME/.local/bin" >> $GITHUB_PATH

    - name: Install requirements
      if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
      run: |
        poetry install

It should theoretically be possible to cache installation of poetry too, which is installed to ~/.poetry. I couldn't find the way, though.

However, by the time you read this, this PR might have already landed so all you're going to need to cache Poetry deps is actions/setup-python

Poetry gets installed to `$HOME/.local/bin`, so you could cache that path and have it available on subsequent runs. — jidicula, Feb 07 '22 at 16:22

How to cache poetry install for GitHub Actions

4 Answers4

Linked