I have installed superset with docker container, logged in to superset. I wanted to connect to Azure Databricks. I have followed the below steps.
I have created "requirements-local.txt" file in /superset/docker/ folder. Added "databricks-dbapi[hive,odbc,sqlalchemy]" in "requirements-local.txt" file.
Ran "docker-compose build --force-rm" command to build.
Ran "docker-compose -f docker-compose-non-dev.yml up" command to run the docker.
Superset is up and running and I am able to login to superset.
Issue:
Go to "Data" -> "Connect a database
"Select "Databricks SQL Endpoint" in "SUPPORTED DATABASES" dropdown.
Provide URL "databricks+pyhive://token:{access_token}@{server_hostname}:{port}/{database_name}" details from Databricks, for "SQLALCHEMY URI" text box
Provide {"connect_args": {"http_path": "sql/protocolv1/o/****"}} with Http_Path, in ADVANCED > Other > ENGINE PARAMETERS text area.
Click on "TEST CONNECTION"
Below ERROR occurred.
ERROR:
superset_app | SupersetErrorsException
superset_app | Traceback (most recent call last):
superset_app | File "/app/superset/databases/commands/test_connection.py", line 102, in run
superset_app | with database.get_sqla_engine_with_context() as engine:
superset_app | File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter
superset_app | return next(self.gen)
superset_app | File "/app/superset/models/core.py", line 372, in get_sqla_engine_with_context
superset_app | yield self._get_sqla_engine(schema=schema, nullpool=nullpool, source=source)
superset_app | File "/app/superset/models/core.py", line 396, in _get_sqla_engine
superset_app | params["poolclass"] = NullPool
superset_app | TypeError: 'str' object does not support item assignment
superset_app |
superset_app | The above exception was the direct cause of the following exception:
superset_app |
superset_app | Traceback (most recent call last):
superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
superset_app | rv = self.dispatch_request()
superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
superset_app | return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
superset_app | File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps
superset_app | return f(self, *args, **kwargs)
superset_app | File "/app/superset/views/base_api.py", line 114, in wraps
superset_app | raise ex
superset_app | File "/app/superset/views/base_api.py", line 111, in wraps
superset_app | duration, response = time_function(f, self, *args, **kwargs)
superset_app | File "/app/superset/utils/core.py", line 1604, in time_function
superset_app | response = func(*args, **kwargs)
superset_app | File "/app/superset/utils/log.py", line 265, in wrapper
superset_app | value = f(*args, **kwargs)
superset_app | File "/app/superset/views/base_api.py", line 84, in wraps
superset_app | return f(self, *args, **kwargs)
superset_app | File "/app/superset/databases/api.py", line 726, in test_connection
superset_app | TestConnectionDatabaseCommand(item).run()
superset_app | File "/app/superset/databases/commands/test_connection.py", line 176, in run
superset_app | raise DatabaseTestConnectionUnexpectedError(errors) from ex
superset_app | superset.databases.commands.exceptions.DatabaseTestConnectionUnexpectedError: [SupersetError(message="'str' object does not support item assignment", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Databricks Interactive Cluster', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
superset_app | 2022-11-29 23:29:08,364:WARNING:superset.views.base:SupersetErrorsException
superset_app | Traceback (most recent call last):
superset_app | File "/app/superset/databases/commands/test_connection.py", line 102, in run
superset_app | with database.get_sqla_engine_with_context() as engine:
superset_app | File "/usr/local/lib/python3.8/contextlib.py", line 113, in enter
superset_app | return next(self.gen)
superset_app | File "/app/superset/models/core.py", line 372, in get_sqla_engine_with_context
superset_app | yield self._get_sqla_engine(schema=schema, nullpool=nullpool, source=source)
superset_app | File "/app/superset/models/core.py", line 396, in _get_sqla_engine
superset_app | params["poolclass"] = NullPool
superset_app | TypeError: 'str' object does not support item assignment
superset_app |
superset_app | The above exception was the direct cause of the following exception:
superset_app |
superset_app | Traceback (most recent call last):
superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
superset_app | rv = self.dispatch_request()
superset_app | File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
superset_app | return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
superset_app | File "/usr/local/lib/python3.8/site-packages/flask_appbuilder/security/decorators.py", line 89, in wraps
superset_app | return f(self, *args, **kwargs)
superset_app | File "/app/superset/views/base_api.py", line 114, in wraps
superset_app | raise ex
superset_app | File "/app/superset/views/base_api.py", line 111, in wraps
superset_app | duration, response = time_function(f, self, *args, **kwargs)
superset_app | File "/app/superset/utils/core.py", line 1604, in time_function
superset_app | response = func(*args, **kwargs)
superset_app | File "/app/superset/utils/log.py", line 265, in wrapper
superset_app | value = f(*args, **kwargs)
superset_app | File "/app/superset/views/base_api.py", line 84, in wraps
superset_app | return f(self, *args, **kwargs)
superset_app | File "/app/superset/databases/api.py", line 726, in test_connection
superset_app | TestConnectionDatabaseCommand(item).run()
superset_app | File "/app/superset/databases/commands/test_connection.py", line 176, in run
superset_app | raise DatabaseTestConnectionUnexpectedError(errors) from ex
superset_app | superset.databases.commands.exceptions.DatabaseTestConnectionUnexpectedError: [SupersetError(message="'str' object does not support item assignment", error_type=<SupersetErrorType.GENERIC_DB_ENGINE_ERROR: 'GENERIC_DB_ENGINE_ERROR'>, level=<ErrorLevel.ERROR: 'error'>, extra={'engine_name': 'Databricks Interactive Cluster', 'issue_codes': [{'code': 1002, 'message': 'Issue 1002 - The database returned an unexpected error.'}]})]
superset_app | 10.48.4.5 - - [29/Nov/2022:23:29:08 +0000] "POST /api/v1/database/test_connection/ HTTP/1.1" 422 296 "http://[Host_IP]:8088/superset/welcome/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"
superset_app | 10.48.4.5 - - [29/Nov/2022:23:29:39 +0000] "GET /api/v1/me/ HTTP/1.1" 200 150 "http://[Host_IP]:8088/superset/welcome/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36"
Please let me know, if I am missing anything while connecting to Azure Databricks.
Please help me to connect to Azure Databricks.
My expectation is to connect superset to Azure Databricks and create chart in superset using Azure databricks data/table.
Thanks
Nagaraj M M
