Stack Overflow
Stack Overflow

Questions tagged [databricks-sql]

Questions about Databricks SQL

For questions about Databricks SQL - a serverless data warehouse on the Databricks Lakehouse Platform that lets you run all your SQL and BI applications at scale with improved performance, a unified governance model, open formats and APIs, and your tools of choice

357 questions
0
votes
1 answer

Delta table not avaialbe from SQL enpoint, but it works well from notebook

I have mounted Azure Data Lake Storage in databricks with: dbutils.fs.mount( source = "abfss://{0}@{1}.dfs.core.windows.net/".format(container_name, storage_account_name), mount_point = mount_path, extra_configs = cfg ) After this I created a…
Hubert
  • 131
  • 8
0
votes
1 answer

Resolve datatype conflicts with Copy command in databricks

I have a requirement to load the data from a csv file in ADLS Gen2 to Delta table. I am using the copy for the same. One of the column in the target table 'IS_ACTIVE' is defined as TINYINT. When triggering the below code, it fails with the following…
shankar
  • 196
  • 14
0
votes
1 answer

Databricks multilevel query on arbitrary condition

I have top level data 'whatever' comprising many (thousands) of records. Each record has data of the form structA structB ... structN, varA varB ... varN each structure may also contain other structures and variables e.g. structAA, structAB...…
Philip Martin
  • 1
0
votes
1 answer

how to store query results to new database and table in databricks

I am a newbie to azure databricks and trying to store query results in the database. I want to create a new database and table on the fly. if it exists then remove existing DB and tables and store query results in the newly created database.…
SeleniumUser
  • 4,065
  • 2
  • 7
  • 30
0
votes
1 answer

How to used a cached table from a different notebook in databricks?

I cached a table in Databricks (SQL Notebook) using CACHE TABLE work_details AS SELECT (....) The problem is that I can only access the cached table if I am in the same notebook. I want to use the table in a different notebook (same cluster) but it…
Martin
  • 55
  • 6
0
votes
1 answer

ParseException when having a field name which contains '#' in SparkSQL

I'm doing a simple operation of inserting some fields of table into another table, both tables are hive tables in databricks, so I'm able to do it with a simple query like: INSERT INTO . (, ) SELECT…
Cristian Ispan
  • 571
  • 2
  • 5
  • 23
0
votes
0 answers

single-line json split

I have a json file which is complicated and I am trying to split it to multiple files. Is there an approach to achieve this My input file {"Ename":"xyz LLC","entity_type":"Third…
user8521874
  • 89
  • 7
0
votes
1 answer

Spark SQL : Get month from single year week column

I have year week column , which is like below format, in pysaprk sql in databricks. input column Year_week 202001 202002 202003 202004 202005 202006 202007 Expected output column Year_week Month 202001 01 202002 01 202003 …
Jha Ayush
  • 67
  • 8
0
votes
2 answers

Delete from adls gen 2 Delta files fails with error

I have requirement where I am deleting duplicate records from delta file using databricks sql. Below is my query %sql delete from delta.`adls_delta_file_path` where code = 'XYZ ' but it gives below…
exploding_data
  • 317
  • 1
  • 14
0
votes
1 answer

Replicating the functionality of CONDITIONAL_TRUE_EVENT (Snowflake) in ANSI SQL to group events Together

I have to rewrite a script written for Snowflake into Databricks and need some help on how to replicate CONDITIONAL_TRUE_EVENT as Databricks doesn't have that function. The need is for me to group events together if they have the same user and…
Andrew Park
  • 29
  • 5
0
votes
1 answer

convert string to date in azure databricks SQL

I have a table with a date column as string data type when i try to retrive data by converting it to date data type getting Null values instead of getting date value Example: select cast(ColumnName as date) from TableName existed data before …
SBC
  • 1
0
votes
1 answer

Unable to attain correct results with CASE WHEN statement in Databricks with Left Join

I am trying to do a left join between two datasets using a CASE WHEN in databricks statement with the following code: ;WITH CTE1 AS ( SELECT *,ROW_NUMBER()OVER(ORDER BY ts_primarysecondaryfocus)RowNum FROM dataverse.accountv41 ),CTE2 AS ( …
Patterson
  • 1,927
  • 1
  • 19
  • 56
0
votes
0 answers

How to cast field as date_type using databricks SQL

The following Databricks SQL code should produce the following date_type outputs for fields: Action-1-Analysing Action-2-Trying-to-meet Action-3-Date-agreed Action-4-Post-meeting Action-5-Chopped The type of date_type output that should appear in…
Patterson
  • 1,927
  • 1
  • 19
  • 56
0
votes
1 answer

Unable to CREATE OR REPLACE VIEW with Databricks using CTE's

I am trying to create a view in Databricks using the WITH method in Databricks with my SQL code. However, I'm getting the error: Error in SQL statement: ParseException: mismatched input '' expecting {'(', 'UP_TO_DATE', 'AS', 'COMMENT',…
Patterson
  • 1,927
  • 1
  • 19
  • 56
0
votes
2 answers

Databricks SQL error: cannot evaluate expression outer in inline table definition

I joined two tables and tried to get a max timestamp value of two timestamp fields. This is my SQL statement: SELECT rts.DataLakeModified_DateTime, rtt.DataLakeModified_DateTime, ( SELECT MAX(DataLakeModifiedDateTime) FROM…
NinjaDev
  • 1,228
  • 11
  • 21
1 2 3
…
23 24