Python and Nvim: How to activate both global and local virtual env at the same time

Question

I'm using coc-pyright in Nvim and running into some issues.

Firstly, in Nvim, running :checkhealth results in the following error message:

## Python 3 provider (optional)
 42   - WARNING: No Python executable found that can `import neovim`. Using the first available executable for diagnostics.
 43   - ERROR: Python provider error:
 44     - ADVICE:
 45       - provider/pythonx: Could not load Python 3:
 46           /home/<project path>/.venv/bin/python3 does not have the "neovim" module. :help provider-python
 47           python3.7 not found in search path or not executable.
 48           python3.6 not found in search path or not executable.
 49           python3.5 not found in search path or not executable.
 50           python3.4 not found in search path or not executable.
 51           python3.3 not found in search path or not executable.
 52           /home/<project path>/.venv/bin/python does not have the "neovim" module. :help provider-python
 53   - INFO: Executable: Not found

This is fine, and makes perfect sense. Adding the pynvim package (pip install pynvim) will result in the following when using :checkhealth:

# Python 3 provider (optional)
 42   - INFO: `g:python3_host_prog` is not set.  Searching for python3 in the environment.
 43   - INFO: Multiple python3 executables found.  Set `g:python3_host_prog` to avoid surprises.
 44   - WARNING: Your virtualenv is not set up optimally.
 45     - ADVICE:
 46       - Create a virtualenv specifically for Neovim and use `g:python3_host_prog`.  This will avoid the need to install the pynvim module in each virtualenv.
 47   - INFO: $VIRTUAL_ENV matches executable
 48   - INFO: Executable: /home/<project path>/.venv/bin/python3
 49   - INFO: Other python executable: /usr/bin/python3
 50   - INFO: Other python executable: /bin/python3
 51   - INFO: Python version: 3.9.5
 52   - INFO: pynvim version: 0.4.3
 53   - OK: Latest pynvim is installed.

This too makes sense. Saves from installing globally. The issue I run into, however, is that this would require me enabling two separate virtual environments at the same time. One for pynvim alone, the other for all local project dependencies, and I can't figure out how I'm supposed to do this.

If I enable the global, coc-pyright can't find any local modules and will throw errors. If I enable the global, then nvim doesn't have pynvim which it needs.

How are you supposed to do this?

Answer 1

The virtualenv that the python_host_prog is pointing to is not for the virtual environment that the python language server uses. It is for nvim to use on python plugins. So you can have a separate virtual environment to use to execute the plugins that has pynvim installed and then your virtual environment for your project.

python3 -m venv ~/venvs/.nvim-venv && source ~/venvs/.nvim-venv/bin/activate && python3 -m pip install pynvim && which python

Then you can take the output from which python and put that in your init.vim as g:python_host_prog and g:python3_host_prog.

Now you can deactivate and go work on a project. Activate your project's virtualenv and start up nvim. You can :checkhealth and see that the python provider is looking at the virtualenv we created earlier. Your Python virtual env provider should be set to the project you are in. The virtualenv provider is the one that I believe coc-pyright uses for providing intellisense and it is also the one that will be used when running !python. So everything will work as intended. Now if you install a python nvim plugin that needs pynvim, it will be available but like @fannheyward said, it is unnecessary for coc-pyright because it doesn't use the python nvim interface that pynvim provides.

It is a bit confusing because nvim will look for a provider to use for itself if one is not explicitly provided and the project virtualenv python is in that resolution chain. So the purpose of the virtual env provider and python provider can easily be confused.

Answer 2

I apologize for misunderstanding your post. I actually had a think on whether or not you meant what I thought you meant before falling asleep last night. So, I do apologize.

I guess I should ask if you've found a solution to this or not?

If you haven't, do you use any pip/venv wrappers like 'pipenv' or 'virtualenvwrapper'(This might be deprecated)? The only reason I ask is that I know 'pipenv' will allow you to actually specify dev deps in your project's 'Pipfile'. So, couldn't you just add neovim and pyright as dev deps to any project you're working on?

If that's not an option 'venv' has a command line argument that allows it access to your system's python packages.

usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
            [--upgrade] [--without-pip] [--prompt PROMPT] [--upgrade-deps]
            ENV_DIR [ENV_DIR ...]

Creates virtual Python environments in one or more target directories.

positional arguments:
  ENV_DIR               A directory to create the environment in.

optional arguments:
  -h, --help            show this help message and exit
  --system-site-packages
                        Give the virtual environment access to the system
                        site-packages dir.
  --symlinks            Try to use symlinks rather than copies, when symlinks
                        are not the default for the platform.
  --copies              Try to use copies rather than symlinks, even when
                        symlinks are the default for the platform.
  --clear               Delete the contents of the environment directory if it
                        already exists, before environment creation.
  --upgrade             Upgrade the environment directory to use this version
                        of Python, assuming Python has been upgraded in-place.
  --without-pip         Skips installing or upgrading pip in the virtual
                        environment (pip is bootstrapped by default)
  --prompt PROMPT       Provides an alternative prompt prefix for this
                        environment.
  --upgrade-deps        Upgrade core dependencies: pip setuptools to the
                        latest version in PyPI

Once an environment has been created, you may wish to activate it, e.g. by
sourcing an activate script in its bin directory.

--system-site-packages might be of some help.

Perhaps install neovim and pyright globally, then when you create your projects, venv, specify this flag. Then point g:python3_host_prog at your newly created venv for your project. Then the venv should have access to all of the packages it needs, the ones in the project and your global 'neovim' and 'pyright'.

Again, I'm sorry for the misunderstanding. That must've come across as super condescending.

If you've found a solution that fits your circumstances, please let me know what you did. And if not, let me know if what I proposed works.

Thanks, and have a good day.

Answer 3

coc.nvim and coc-pyright doesn't need pynvim to work, you don't need to install this module.