Summary
I'm unable to install GX on a Databricks Notebook (using Python). As I'm stuck at the very first step of the guide, I'm unable to proceed with GX on DX. Any kind of help would be appreciated!
Environment
I'm trying to install GX in a Databricks notebook's scope as suggested in the docs here. The notebook is attached to a cluster with the following specs:
- 1 Driver, 14 GB Memory, 4 Cores
- Runtime: 11.3.x-scala2.12
- Python: 3.9.5
Steps to reproduce
The installation step:
%pip install great-expectations
Finished with the following:
Can't uninstall 'nbconvert'. No files were found to uninstall.
Attempting uninstall: notebook
Found existing installation: notebook 6.4.5
Not uninstalling notebook at /databricks/python3/lib/python3.9/site-packages, outside environment /local_disk0/.ephemeral_nfs/envs/pythonEnv-dc5a75f8-a385-41ce-9ccc-21e19faa4b1c
Can't uninstall 'notebook'. No files were found to uninstall.
Successfully installed altair-4.2.2 anyio-3.7.0 beautifulsoup4-4.12.2 colorama-0.4.6 exceptiongroup-1.1.1 fastjsonschema-2.17.1 great-expectations-0.16.15 importlib-metadata-6.6.0 jinja2-3.1.2 jsonpatch-1.32 jsonpointer-2.3 jupyter-client-8.2.0 jupyter-core-5.3.0 jupyter-events-0.6.3 jupyter-server-2.6.0 jupyter-server-terminals-0.4.4 makefun-1.15.1 marshmallow-3.19.0 mistune-2.0.5 nbclassic-1.0.0 nbconvert-7.4.0 nbformat-5.9.0 notebook-6.5.4 notebook-shim-0.2.3 overrides-7.3.1 python-json-logger-2.0.7 pyyaml-6.0 pyzmq-25.1.0 rfc3339-validator-0.1.4 rfc3986-validator-0.1.1 ruamel.yaml-0.17.17 ruamel.yaml.clib-0.2.7 sniffio-1.3.0 soupsieve-2.4.1 tinycss2-1.2.1 toolz-0.12.0 tornado-6.3.2 tqdm-4.65.0 traitlets-5.9.0 tzlocal-5.0.1 websocket-client-1.5.2
Python interpreter will be restarted.
There is also a warning attached to the cell saying:
Core Python package version(s) changed: The version of a core Python package, e.g., IPython or one of IPython's dependencies, changed: traitlets: 5.1.0 -> 5.9.0. Some Python notebook and PySpark features may no longer work as expected. If you see issues, you can reset the environment by detaching and re-attaching the notebook. Some notebook features may stop working as expected.
Next, I tried the following step, namely:
import great_expectations as gx
from great_expectations.checkpoint import SimpleCheckpoint
which returned:
DynamicAssetError: Updating forward references for asset model ClipboardAsset raised TypeError: issubclass() arg 1 must be a class
