Stack Overflow
Stack Overflow

Questions tagged [databricks-sql]

Questions about Databricks SQL

For questions about Databricks SQL - a serverless data warehouse on the Databricks Lakehouse Platform that lets you run all your SQL and BI applications at scale with improved performance, a unified governance model, open formats and APIs, and your tools of choice

357 questions
0
votes
2 answers

How to convert the below T-SQL Query ISNULL(NAME,'N/A') to Spark-SQL Equivalent

Not able to convert the below T-SQL Query part ISNULL(NAME,'N/A') to Spark-SQL Equivalent SELECT ID, ISNULL(NAME,'N/A') AS NAME, COMPANY FROM TEST to convert the below T-SQL Query part ISNULL(NAME,'N/A') to Spark-SQL…
Nagesh Rudraiah
  • 5
  • 3
0
votes
2 answers

Spark CASE WHEN/When Otherwise that doesn't stop evaluating and accumulates

Say I have following table/dataframe: Id Col1 Col2 Col3 1 100 aaa xxx 2 200 aaa yyy 3 300 ccc zzz I need to calculate an extra column CalculatedValue which could have one or multiple values based on other columns' values. I have…
Martin
  • 78
  • 5
0
votes
1 answer

Previous month query - Databricks

I try to find a function where I can extract the result of the last month only (for exemple if I launch the query in november, I want to display only the resultat of october) There the result : I dont know if I have to enter the function in my…
shosho88
  • 31
  • 2
0
votes
1 answer

Need help on sub-query of Spark-SQL Databricks

I have below mentioned SQL and getting below mentioned dataset as result. But i want to display only one Open status record which has MIN date. SELECT distinct o.svc_ord_nbr AS SVC_ORD_NBR, o.svc_ord_stat_nm AS SVC_ORD_STAT_NM, …
Krishna
  • 35
  • 5
0
votes
1 answer

PySpark Delete Specific Rows Without Primary Key

I am trying to make up for PySpark's lack of 'WHEN NOT MATCHED BY SOURCE' in MERGE statements. I want to delete rows from a table when there are no matching rows in another table. However, my table doesn't have a primary key, so I'm struggling to…
Daniel
  • 73
  • 8
0
votes
1 answer

Add Column to Databricks Table Syntax Error

I am trying to add a column based on a case when statement mapping status to status names. This is in Databricks, and i'm wondering if i'm having an issue with permissions or if my syntax is off somewhere. A similar statement worked in SQL server. …
CharlieBitMaFinga
  • 35
  • 6
0
votes
0 answers

Cursor alternative in databricks notebook (sql,scala)

I need to migrate a certain table using databricks notebooks. Problem is databricks SQL doesn't support the implementation of cursors. What alternative could I use? This is roughly what I need to do, for reference: ` Declare @GrupoReg as…
Finatsu
  • 1
  • 1
0
votes
2 answers

AnalysisException: Multi-column In predicates are not supported in the DELETE condition

I am doing the following in my databricks code. The code is trying to delete some rows from my table under conditions which seam simple to me. from datetime import datetime, timedelta today = datetime.today() yesterday = today - timedelta(days =…
Ali Saberi
  • 864
  • 1
  • 10
  • 33
0
votes
1 answer

Calculate working days between two dates: null value returned

I'm trying to figure out the number of working days between two dates. The table (dfDates) is laid out as follows: Key StartDateKey EndDateKey 1 20171227 20180104 2 20171227 20171229 I have another table (dfDimDate) with all the…
Carolina Karoullas
  • 360
  • 1
  • 14
0
votes
2 answers

Column names appearing as record data in Pyspark databricks

I'm working on Pyspark python. I downloaded a sample csv file from Kaggle (Covid Live.csv) and the data from the table is as follows when opened in visual code (Raw CSV data only partial…
Mahesh M
  • 69
  • 1
  • 10
0
votes
1 answer

Databricks (SQL Warehouse) How to read data from postgres (jdbc)

Trying to connect database sql warehouse from postgres db but the problem I am facing is that it shows me that data source is not supported. Please see screenshot below. enter image description here
user8536590
  • 1
  • 2
0
votes
1 answer

How to delete rows efficiently in sparksql?

I get a view with IDs for which I have to delete the corresponding records in a table present in a database. View: |columnName| ------------ | 1 | | 2 | | 3 | | 4 | Variables: tableName = name of the table columnName =…
Martin
  • 55
  • 6
0
votes
1 answer

How to select only completed months of data where date is in format YYYY-MM?

I have created a query to generate some dates: create or replace temporary view dates as select date_format(add_months(now(), -2), 'y-MM') as date union select date_format(add_months(now(), -1), 'y-MM') as date union select…
Chris Snow
  • 23,813
  • 35
  • 144
  • 309
0
votes
1 answer

Databricks SQL mismatch issue with T-SQL

When I appy the following query using T-SQL on MS Server I don't have any problems: SELECT DISTINCT * FROM dbo.account LEFT OUTER JOIN dbo.crm2cburl_lookup ON account.Id = CRM2CBURL_Lookup.[Key] LEFT OUTER JOIN dbo.organizations ON…
Patterson
  • 1,927
  • 1
  • 19
  • 56
0
votes
1 answer

How to display recent timestamp records and ignore the previous received timestamp records in SQL(databricks)

select * from XYZ, the below shows the result set. timestamp | Name 20221006050109 | A 20221006050109 | B 20221006050109 | C 20221006050109 | D 20221007050125 | E 20221007050125 | F 20221007050125 | …
nitin sinha
  • 3
  • 2
1 2 3
…
23 24