Stack Overflow
Stack Overflow

Questions tagged [databricks-connect]

172 questions
0
votes
1 answer

Databricks with python 3 for Azure SQl Databas and python

I am trying to use Azure Databricks in order to : 1- insert rows into table of Azure SQL Databse with python 3. I cannot see a documentation about insert rows. (I have use this link to connect to the database Doc and it is working). 2- Save Csv…
hugo
  • 441
  • 5
  • 25
0
votes
1 answer

Spark session is not getting initialized | sparkR.session() gives the error "Error in if (len > 0) { : argument is of length zero"

I am currently using Rstudio server hosted outside databricks cluster and followed the steps to configure Databricks Connect. The connection test was also successful . But when I initialize a spark session using the below code, it throws up an…
R__
  • 51
  • 3
0
votes
1 answer

Databricks Connect is not yet supported on the cluster with process isolation enabled

I'm using Azure Databricks (6.5 Runtime) and trying to hook it up to an IDE. However, in order to enable AAD credential passthrough in our cluster we need to set the following properties to true spark.databricks.passthrough.enabled…
Saravanan
  • 49
  • 6
0
votes
1 answer

Having trouble connecting to Databricks CLI

so I need to connect to Databricks CLI so that I could download a dbfs jar file on my local machine. To do that, I am using Databricks CLI. I tried to use this command and here's the message I got. Could you please explain why I'm getting this error…
jukebox
  • 453
  • 2
  • 8
  • 24
0
votes
1 answer

How do I connect to and write a csv file to a remote instance of Databricks Apache Spark from Java?

I'm trying to connect to a remote instance of Databricks and write a csv file to a specific folder of the DBFS. I can find bits and pieces here and there but I'm not seeing how to get this done. How do I add the file to DBFS on a remote Databricks…
John
  • 3,458
  • 4
  • 33
  • 54
0
votes
0 answers

databricks-connect not working on windows

I've installed databricks-connect on Windows 10 with the instructions here: https://docs.databricks.com/dev-tools/databricks-connect.html After running databricks-connect configure and entering all values, i'm running databricks-connect test. This…
Omri374
  • 2,555
  • 3
  • 26
  • 40
0
votes
1 answer

Azure Data brick connection using databricks-connect

i am following https://docs.databricks.com/dev-tools/databricks-connect.html to connect with azure databricks #creating environment dbconnect (base) C:\>conda create --name dbconnect python=3.7 (base) C:\>conda activate dbconnect (dbconnect)…
qaiser
  • 2,770
  • 2
  • 17
  • 29
0
votes
1 answer

I want to fetch count (*) for all tables in a DB at a time along with 'Table_Name' in DataBricks SQL

For example, I want output something like this in Databricks. I know how to do it in SSMS or Oracle but that syntax is not working in Databricks. I tried using the 'show tables' but its not serving my purpose.
ashwini bulange
  • 9
  • 3
0
votes
1 answer

KNIME Databricks community connection

I've been using Databricks Community cluster Environment from Knime for weeks. I don't know why now it's not possible and an unexpected error message appears in the console log of Knime. ERROR Create Databricks Environment 2:1 Execute failed:…
Julián Gómez Cuadrado
  • 23
  • 4
0
votes
1 answer

Permission denied while inserting data from Azure Databricks to Synapse in production environment

We all have a scenario in our project where we are inserting data from Databricks dataframes into Azure Synapse. While we could do this without issues on Dev environment with admin access, we could not run this in higher environment. On Higher…
TANIMA DUTTA
  • 1
  • 1
0
votes
1 answer

Connection Error - Reading Data From Snowflake Into Databricks

I am trying to connect to snowflakes from databricks. I tried number of methods but without any success. Method I # snowflake connection options options = dict(sfUrl="eu-west-1.snowflakecomputing.com", sfUser= "snowflake_usr", …
James Taylor
  • 484
  • 1
  • 8
  • 23
0
votes
1 answer

Spark implicits do not work on dbutils object

Why cannot I do somthing like this? import spark.implicits._ dbutils.fs.ls(folderName).toDF throws an error of value toDF is not a member of Seq[com.databricks.backend.daemon.dbutils.FileInfo] But it works fine in databricks notebook. Can anyone…
Srinivas
  • 2,010
  • 7
  • 26
  • 51
0
votes
1 answer

Databricks SQL Server connection using integrated authentication

I'm trying to connect my Databricks cluster to an existing SQL Server database using python. I will like to leverage the integrated authentication method. Getting error com.microsoft.sqlserver.jdbc.SQLServerException: This driver is not configured…
jgtrz
  • 365
  • 6
  • 19
0
votes
1 answer

Databricks Notebook - Microsoft Azure - Autocompletion is NOT working when attached to cluster

Databricks Notebook - Microsoft Azure - Autocompletion is NOT working when databricks notebook is attached to cluster. Can someone please share any pointers to fix this? [Edited] I have a couple of clusters, and i see this behaviour of autocomplete…
Raja
  • 513
  • 5
  • 18
0
votes
0 answers

Databricks-connect: sparkContext.wholeTextFiles

I have setup databricks-connect version 5.5.0. This runtime includes Scala 2.11 and Spark 2.4.3. All the Spark code I have written has been correctly executed and without any issues until I tried calling sparkContext.wholeTextFiles. The error that I…
zaxme
  • 1,065
  • 11
  • 29
1 2 3
…
11
12