2

I am trying to run some code using databricks-connect, but I am suddenly running into this error,

22/06/24 13:24:48 ERROR SparkClientManager: Fail to get the SparkClient
java.util.concurrent.ExecutionException: com.databricks.service.SparkServiceConnectionException: Invalid shard address

Everything was working fine until today. It looks like some of my colleagues are getting the same error as well, out of nowhere.

I came across this post as well, but it only provides the link to an article that explains how to set up db-connect on VS Code,
https://community.databricks.com/s/question/0D53f00001fckBSCAY/databricksconnect-invalid-shard-address

How can this be resolved?

Nikunj Kakadiya
  • 2,689
  • 2
  • 20
  • 35
Minura Punchihewa
  • 1,498
  • 1
  • 12
  • 35
  • The answer given by @Alex Ott solved this issue, however, I also found this article in the Microsoft documentation that may be of use, https://learn.microsoft.com/en-us/azure/databricks/kb/dev-tools/dbconnect-protoserializer-stackoverflow – Minura Punchihewa Jun 24 '22 at 15:42

2 Answers2

4

That's issue that did arise recently due the way how the host is parsed - to fix it, remove trailing slash (/) from the Databricks workspace host. I.e., https://adb-131442342.9.azuredatabricks.net/ should be https://adb-131442342.9.azuredatabricks.net

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
0

Everything was working fine until today.

If it was working fine then this should be some internal network issue which you try to resolve by re-routing the private endpoints and also you can try to restart the cluster on databricks-cli using below command:

databricks clusters restart --cluster-id 0802-090441-honks846

Utkarsh Pal
  • 4,079
  • 1
  • 5
  • 14