So first a bit of context...
I'm trying to set up a JupyterLab environment where I can (reasonably seamlessly) change between Python kernels to assess code performance. Target kernels would be standard CPython (Python 3 - mainly latest versions) and PyPy. I'm using conda to manage my environments.
I have set up a JupyterLab environment (installing JupyterLab in base conda environment) and by creating various conda environments based on different CPython versions (in which I have install ipykernel as well), I can easily change run time kernel in JupyterLab. So far so good.
So, to complete the endeavour, I just need to create a PyPy environment in conda (as per PyPy instructions) and install ipykernel in it. It appears impossible to do this.
My expectation is that I create a PyPy environment as per PyPy instructions, add ipykernel to it and I will be able to switch to a PyPy based kernel in JupyterLab (as I have successfully done with CPython environments). My expectations are not met!
I start by creating a PyPy environment in conda:
conda config --set channel_priority strict
conda create -c conda-forge -n pypy-env pypy python=3.8
It takes a while, but appears to work:
(base) PS C:\Windows\system32> conda env list
# conda environments:
#
base * C:\tools\miniconda3
py-311 C:\tools\miniconda3\envs\py-311
pypy-env C:\tools\miniconda3\envs\pypy-env
(base) PS C:\Windows\system32> conda activate pypy-env
(pypy-env) PS C:\Windows\system32> python --version
Python 3.8.16 | packaged by conda-forge | (a9dbdca6, May 11 2023, 16:56:34)
[PyPy 7.3.11 with MSC v.1929 64 bit (AMD64)]
Attempting to add ipykernel (necessary to enable kernel switching in JupyterLab) fails after quite a spirited attempt and highlights MANY conflicts (abbreviated for convenience):
(pypy-env) PS C:\Windows\system32> conda install ipykernel
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: -
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package ca-certificates conflicts for:
pip -> python[version='>=2.7,<2.8.0a0'] -> ca-certificates
python=3.8 -> openssl[version='>=3.0.8,<4.0a0'] -> ca-certificates
...
...
Package six conflicts for:
ipykernel -> packaging -> six
pip -> html5lib -> six[version='>=1.9']
Note that strict channel priority may have removed packages required for satisfiability.
Although advised by PyPy, the strict channel priority looks to be causing issues. Setting it back from strict to the default flexible
(pypy-env) PS C:\Windows\system32> conda config --set channel_priority flexible
(pypy-env) PS C:\Windows\system32>
enables seemingly satisfactory installation:
(pypy-env) PS C:\Windows\system32> conda install ipykernel
Collecting package metadata (current_repodata.json): done
Solving environment: done
...
...
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Retrieving notices: ...working... done
However, starting JupyterLab (JL) and selecting the new pypy-env kernel fails, as the JL logs confirm:
CONDA_PREFIX=C:\tools\miniconda3
C:\tools\miniconda3\envs\pypy-env\lib\site-packages\jupyter_client\__init__.py:23: UserWarning: Could not import submodules
warnings.warn("Could not import submodules")
Traceback (most recent call last):
File "C:\tools\miniconda3\envs\pypy-env\Lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\tools\miniconda3\envs\pypy-env\Lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\tools\miniconda3\envs\pypy-env\lib\site-packages\ipykernel_launcher.py", line 15, in <module>
from ipykernel import kernelapp as app
File "C:\tools\miniconda3\envs\pypy-env\lib\site-packages\ipykernel\__init__.py", line 5, in <module>
from .connect import * # noqa
File "C:\tools\miniconda3\envs\pypy-env\lib\site-packages\ipykernel\connect.py", line 12, in <module>
from jupyter_client import write_connection_file
ImportError: cannot import name 'write_connection_file' from 'jupyter_client' (C:\tools\miniconda3\envs\pypy-env\lib\site-packages\jupyter_client\__init__.py)
Changing to the JL kernel to a CPython conda environment (aside from a few warnings in the JL log) appears to work fine (and executes code correctly in the notebook):
CONDA_PREFIX=C:\tools\miniconda3
This version of python seems to be incorrectly compiled
(internal generated filenames are not absolute).
This may make the debugger miss breakpoints.
Related bug: http://bugs.python.org/issue1666807
[I 2023-06-30 18:35:24.069 ServerApp] Connecting to kernel 834d68cd-6153-4f99-9beb-ded2e414bc14.
[I 2023-06-30 18:35:24.088 ServerApp] Connecting to kernel 834d68cd-6153-4f99-9beb-ded2e414bc14.
[I 2023-06-30 18:35:24.111 ServerApp] Connecting to kernel 834d68cd-6153-4f99-9beb-ded2e414bc14.
So, in summary, following PyPy's advice of strict conda channel priority makes it impossible to install ipykernel. Ignoring PyPy's advice regarding conda channel priority and sticking with the flexible default enables what appears to be a successful install but in fact is inoperable (at least in the context of ipykernel and JupyterLab).
Anyone have any suggestions how to rectify this situation and enable a functional install of PyPy that works with ipykernel and JupyterLab?
