0

Can you please guide me on where I can find the logs of the Library Installation in Azure Databricks ? I am trying to install the spark-sql_2.11 package from Maven which is failing and there are no details on why it is failing. It would be great if someone can help identify the issue.

Steps to Install:

   Clusters -> Libraries -> Install New 
-> Maven Tab -> Search Packages 
-> Select Maven Central Repository in Drop Down -> Search for spark-sql_2.11 
-> Select compatible Spark version in release i.e. **2.4.5** in my case 
-> Click Select and Install.

I am facing the same issue with spark-hive_2.11 (2.4.5 release) but installation of avro-tools from maven is working as expected. Not sure what I am missing.

Thanks for all the help.

rainingdistros
  • 450
  • 3
  • 11
  • Doesn't cluster already comes with ` spark-sql` ? – koiralo Jun 19 '20 at 09:48
  • I was not able to find any information regarding that...I did go through the libraries installed by default in the below link, but I could not find any reference. https://docs.databricks.com/release-notes/runtime/6.6.html Besides it is just not this library, I have a had a couple of libraries from other Maven repositories fail during their installations. It would be great if I can find why they are failing, maybe because of dependencies. Would be great if I can find a log telling me what went wrong. It just goes to resolving state and then fails without any other information. – rainingdistros Jun 21 '20 at 15:13

0 Answers0