2

I'm using databricks-connect on mac using pycharm but after I finished the configuration and tried to run databricks-connect test, I got the following error and have no idea what the problem is. I followed this documentation: https://docs.databricks.com/user-guide/dev-tools/db-connect.html

The error message is as below:

scala> spa
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/bin/databricks-connect", line 11, in 
    load_entry_point('databricks-connect==5.3.1', 'console_scripts', 'databricks-connect')()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyspark/databricks_connect.py", line 244, in main
    test()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyspark/databricks_connect.py", line 213, in test
    raise ValueError("Scala command failed to produce correct result")
ValueError: Scala command failed to produce correct result
Alex Ott
  • 80,552
  • 8
  • 87
  • 132
efsee
  • 579
  • 1
  • 10
  • 22

3 Answers3

1

Maybe your Java/Python version does not comply. Check your cluster, which Python version does it use (in my case it was 3.5).

And what's most important: check which JDK version do you have on your computer. In my case, I've had the latest one, which was not supported by databricks-connect. It needs to run on JDK 8.

stang
  • 46
  • 3
1

I would make sure you are using the right version of the Databricks Runtime (DB Connect only currently supports 5.1-5.5). Since these is a limit on the DBR that works with DB connect, you'll have to make sure you match the python version as well (for the base Databricks runtime, I believe it is 3.5.x).

Molly
  • 49
  • 4
0

To ignore the RUNTIME version, export an environment variable that resolves:

export DEBUG_IGNORE_VERSION_MISMATCH=1

chá de boldo
  • 26
  • 3