Poetry already added module, but not installed

Question

My pyproject.toml is something like this

[tool.poetry.dependencies]
python = "^3.8"
numpy = "^1.19.4"
pandas = "^1.1.5"
matplotlib = "^3.3.3"
seaborn = "^0.11.0"
lightgbm = "^3.1.1"
jupyter = "^1.0.0"
notebook = "^6.1.5"
.
.
.
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

After poetry install and open python, I found I could import some module(numpy and pandas), but not others like seaborn, lightgbm and etc.. Anyone knows how to fix this problem? I got same situation, so I manually deleted .venv and poetry.lock. Then I re-added modules by poetry install. But it didn't go well...

$ poetry run python
Python 3.9.1 (default, Dec 10 2020, 10:36:35)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

>>> import numpy
>>> import pandas

>>> import seaborn
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'seaborn'

>>> import lightgbm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'lightgbm'

I guess llvmlite is a main cause of this problem, so I'll work on looking around for solution. But if you already know about it, please share.

$ poetry cache clear pypi --all
$ rm -r .venv
$ rm poetry.lock
$ poetry install

### a lot of process goes on

  • Installing colorama (0.4.4): Installing...
  • Installing llvmlite (0.35.0rc3): Failed
  • Installing llvmlite (0.35.0rc3): Failed
  • Installing colorama (0.4.4)
  • Installing llvmlite (0.35.0rc3): Failed
  RuntimeError
  Unable to find installation candidates for llvmlite (0.35.0rc3)
  at /usr/local/Cellar/poetry/1.1.4/libexec/lib/python3.9/site-packages/poetry/installation/chooser.py:72 in choose_for
       68│
       69│             links.append(link)
       70│
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│
       76│         # Get the best link

Followings are possible clues to solve

poetry replies it is already installed.

$ poetry add seaborn
The following packages are already present in the pyproject.toml and will be skipped:

  • seaborn

If you want to update it to the latest compatible version, you can use `poetry update package`.
If you prefer to upgrade it to the latest available version, you can use `poetry add package@latest`.

Nothing to add.

$ poetry run which python

/Users/<user>/.venv/bin/python

$ poetry env info

Virtualenv
Python:         3.9.1
Implementation: CPython
Path:           /Users/<user>/.venv
Valid:          True

System
Platform: darwin
OS:       posix
Python:   /usr/local/Cellar/python@3.9/3.9.1/Frameworks/Python.framework/Versions/3.9

$ poetry -V

Poetry version 1.1.4

Answer 1

I fixed this problem. This problem is just about failure of installing llvmlite, rather than poetry. llvmlite is pre-requisite for installing pandas-profiling I was trying to install.

Rule of thumb when it comes to finding dependence problem

1. removing current environment

$ poetry cache list
# after finding out what cache exists
$ poetry cache clear <cache name> --all
$ rm -r .venv
$ rm poetry.lock 

2. Take a note or copy of what modules you have installed

$ cat pyproject.toml
# take a note or copy of what modules you have installed

$ rm pyproject.toml

3. re-initialize environment and adding modules incrementally

$ poetry init
# not adding modules except ones you are sure not to cause problem

$ poetry add <module>

4. Identifying what is actually problem

Answer 2

For me, I had a super-specific case, but it happened basically because direnv tool applied a different venv beforehand which mixed up everything.