0

I'm using Azure Databricks (6.5 Runtime) and trying to hook it up to an IDE. However, in order to enable AAD credential passthrough in our cluster we need to set the following properties to true

spark.databricks.passthrough.enabled true
spark.databricks.pyspark.enableProcessIsolation true

But this causes databricks-connect test to fail with the error

Caused by: java.lang.IllegalArgumentException: requirement failed: Databricks Connect is not yet supported on the cluster with process isolation enabled

Are there any workarounds for this? If not, then are there plans to support this use case anytime soon?

Saravanan
  • 49
  • 6

1 Answers1

2

Make sure the below configuration is set to true.

 spark.databricks.libraryIsolation.processIsolation.enabled = true
CHEEKATLAPRADEEP
  • 12,191
  • 1
  • 19
  • 42
  • If it is useful for you, could you please [accept it as an answer](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work)? It may help more people who have similar issue. – CHEEKATLAPRADEEP Nov 06 '20 at 04:22
  • no that doesn't work, when creds passthrough is enabled on HC cluster it enables processIsolation, the databricks-connect library is not yet compatiblie with that. – Geethanadh Mar 03 '21 at 18:50