3

I am running a custom-configured Jupyterlab installation for our company. It is wired to Livy and Spark to provide PySpark and Spark kernels, which works fine.

However, there is still the default Python 3 Kernel I want to remove: Kernels

When dropping to a console into the pod and doing a jupyter kernelspec list, I get, as expected:

Available kernels:
  pysparkkernel    /usr/local/share/jupyter/kernels/pysparkkernel
  python3          /usr/local/share/jupyter/kernels/python3
  sparkkernel      /usr/local/share/jupyter/kernels/sparkkernel

When I jupyter kernelspec remove python3, and reload Jupyterlab, the Python 3 Kernel is still there, and it reappears in the list:

Available kernels:
  python3          /usr/local/lib/python3.8/dist-packages/ipykernel/resources
  pysparkkernel    /usr/local/share/jupyter/kernels/pysparkkernel
  sparkkernel      /usr/local/share/jupyter/kernels/sparkkernel

Even when I delete this /usr/local/lib/python3.8/dist-packages/ipykernel/resources dir the kernel doesn't disappear.

Is it even possible to disable this default kernel or do I have to live with it?

Workaround

I got what I wanted by copying everything from the pysparkkernel into the python3 dir and removing the pysparkkernel, effectively making Jupyterlab think that the PySpark kernel is the default one.

While this works, I am still interested if this is the only way.

rabejens
  • 7,594
  • 11
  • 56
  • 104

1 Answers1

3

In addition to removing the kernel spec you need to turn off ensure_native_kernel option which makes sure that the default kernel is always included by adding:

c.KernelSpecManager.ensure_native_kernel = False

to your configuration file. You can set this via command line too:

jupyter lab --KernelSpecManager.ensure_native_kernel=False

as explained in https://github.com/jupyter/notebook/issues/3674

Note: if you are using JupyterLab 3.x with jupyter_server, the configuration is now stored in jupyter_server_config.py rather than jupyter_notebook_config.py as explained in the migration guide.

krassowski
  • 13,598
  • 4
  • 60
  • 92
  • I am generating the config on the fly when first starting Jupyterlab to be able to configure things with environment variables put into the Kubernetes deployment file. I will try this option. – rabejens Jan 29 '21 at 23:08
  • Unfortunately, it did not work. I added the line to my config but after removing the default kernel it is still there. I am using Jupyterlab with Jupyterhub though. – rabejens Feb 01 '21 at 09:38
  • Are you sure that you are editing the correct configuration file? JupyterHub has not switched to jupyter_server yet (although it may change with JupyterLab 3.0.6) – krassowski Feb 01 '21 at 11:35