Using ipyleaflet within a Vertex AI Managed Notebook running on a Docker image

Question

TL;DR How does one get ipyleaflet to work in a Vertex AI Managed Notebook booted from a custom Docker image?

Objective

Following on this thread, I am working in JupyterLab within a Managed Notebook instance, accessed through the Vertex AI workbench, as part of a Google Cloud Project. I am trying to supply a custom Docker image, such that when a Jupyter Lab notebook is launched from the running instance, it (a) contains some modules for performing and visualizing cartographic analysis, and, importantly, (b) permits visualization with ipyleaflet and associated modules.

What I've tried

Thus far, I have succeed in creating a Docker image (derivative of Google image) that supplies, in the running Jupyter Lab, a dedicated environment (explicitly exposed kernel) with the correct modules (in particular, geemap, earthengine-api, ipyleaflet, ipywidgets). The modules are all importable and appear sound. However, so far as I can tell, supplying a custom Docker image during the build process, effectively breaks the ipyleaflet (and presumably widgets, events, etc) connection that Google's Jupyter Lab base image has if one creates a Managed Notebook without supplying a Docker image. Attempts to create map visualizations returns, "Error displaying widget: model not found", discussed at 1, 2, 3, 4. In other words, if one creates a Managed Notebook without a Docker image, starts the notebook instance, launches Jupyter Lab, opens a notebook, and then uses %pip install xyz for the modules of interest, ipyleaflet-based mapping works fine. I suspect that the nuance of difference here, is that the latter method (%pip installing from within the notebook), is being layered on top of a fully formed base Jupyter Lab container (per @gogasca's comment here, that the Google Managed Jupyter Lab runs as a container that is not customizable).

Questions

So, what I would love to know is:

(1) How does one retain ipyleaflet (and associated modules) functionality in a Managed Notebook that is based on a user supplied Docker image?

(2) Is there a way to effectively replicate the %pip install approach when using a custom Docker image, such that commands specified in the Docker file are layered on top of a fully formed base Google image.

To question (2), I suspect that when the gcloud sdk for Managed Notebooks is available (currently under the impression that this is a work in progress), it will be possible to provide a post-startup-script, as in this example. I am aware that there are REST and Terraform build options available that might satisfy my Managed Notebook needs. These require quite a bit more legwork though, so I am interested in simpler solutions, if they exist.

EDIT

Adding content for partial reproducibility. Steps to reproduce:

1. A custom Docker image was created using the interface at https://shell.cloud.google.com/.
2. Within the cloud shell, from the terminal, set gcloud configuration parameters with gcloud config set project yourProjectIDHere
3. Create a file named Dockerfile with the following content:

FROM python:3.7.4-buster 
ENV VIRTUAL_ENV=/env/testEnvironment 
RUN python3 -m venv $VIRTUAL_ENV --system-site-packages 
ENV PATH="$VIRTUAL_ENV/bin:$PATH" 
# Install dependencies: 
COPY requirements.txt . 
RUN $VIRTUAL_ENV/bin/pip install -r requirements.txt

4. Create a file named requirements.txt with the following content

ipython
ipykernel
geemap
earthengine-api
ipyleaflet
folium
voila
ipywidgets

5. From the cloud shell terminal, build your Docker image (this takes a long time to build. We will use a Google deep learning library derivative once we get our workflow sorted): docker build . -f Dockerfile -t "gcr.io/yourProjectIDHere/test-image:latest"
6. Push container to GCP Container Registry docker push "gcr.io/yourProjectIDHere/test-image:latest"
7. Move to Google cloud console and create a new Managed Notebook within your GCP.
8. Select the defaults except for (a) use Service account permissions; (b) use "Networks shared with me" associated with the host project, including both a Network and Shared subnetwork (presuming that this isn't going to change installation components, so probably doesn't matter); (c) uncheck "Enable external IPs"; (d) check "Enable terminal"; and (e) check "Provide custom docker images".
9. From the "Provide custom docker images" dialog, select the image you just created.
10. Create the Managed Notebook, and once created, open Jupyter Lab.
11. The Docker image should expose a kernel in the Jupyter Lab environment. Open a new notebook using the kernel.
12. Load and test modules within the notebook

# Import and initialize the earthengine-api by whatever means fit your use case
import ee
ee.Initialize( ... )
# Test authentication pathways
print(ee.Image("NASA/NASADEM_HGT/001").get("title").getInfo())
# Provided you have your authentication set-up correctly...

# Test import of various other modules   
import os
import sys
import pandas as pd
import numpy as np
import geemap
from IPython.display import display, HTML, Image
import ipywidgets as widgets
from ipywidgets import Layout
from ipywidgets import interact, interactive, fixed, interact_manual, Button, HBox, VBox
from ipywidgets import HTML
import voila

# Test inline mapping using ipyleaflet-based map     
Map = geemap.Map(center=(-0.2557968807155925, 119.46629773460036), zoom=5)
Map

# Receive error.

# This is also replicable without earthengine-api or geemap, by just trying to make a basic ipyleaflet map
from ipyleaflet import Map, basemaps, basemap_to_tiles
m = Map(basemap=basemap_to_tiles(basemaps.OpenStreetMap.Mapnik), center=(48.204793, 350.121558), zoom=3)
m