Stack Overflow
Stack Overflow

Questions tagged [databricks-sql]

Questions about Databricks SQL

For questions about Databricks SQL - a serverless data warehouse on the Databricks Lakehouse Platform that lets you run all your SQL and BI applications at scale with improved performance, a unified governance model, open formats and APIs, and your tools of choice

357 questions
-1
votes
1 answer

Pull code from on prem database to create AZURE SQL table including indexes, constraints, keys, etc. using pyspark

I need to fetch the 'CREATE TABLE' code with all primary keys,indexes, collation etc. from on prem SQL Server db to Databricks and then from Databricks to Azure SQL db. I have already connection to both SQL Server databases from Databricks. I'm…
Hanna
  • 5
  • 3
-1
votes
1 answer

interactive cluster vs SQL warehouse to connect to Power BI

I was reading about how to connect Power BI with Databricks and found this article from Microsoft and one of the requirements is: An Azure Databricks cluster or Databricks SQL warehouse So, I would like to know which one is the best option in…
BryC
  • 89
  • 6
-1
votes
1 answer

Execute query in parallel over a list of rows in pyspark

In databricks I have N delta tables of stores with their products with this schema: store_1: store product sku 1 prod 1 abc 1 prod 2 def 1 prod 3 ghi store_2: store product sku 2 prod 1 abc 2 prod 10 xyz 2 prod…
Andrés Bustamante
  • 442
  • 1
  • 4
  • 15
-1
votes
1 answer

Efficient SQL Query on Hive for max value

I have the following table of data (as Delta table, which is mapped as Hive table) UtilEvents: ----------------------------------------------------------------------------- SerialNumber EventTime UseCase RemoteHost …
Ganesha
  • 145
  • 1
  • 10
-1
votes
1 answer

SQL query in Databricks to return the value of subtraction of time?

From the following table, my goal is to fill in the value in the column "clmnTimeDiff". Suppose that I have clmnTime1 and clmnTime2, which store the value of "TIME" in string, I want to let the field clmnTimeDiff store the difference between…
Sunkung Krp
  • 13
  • 1
-1
votes
1 answer

Timestamp comparison is failing in spark SQL in databricks

I was executing below simple Spark-SQL code azure databricks. val df2=spark.sql( s""" select mbrgm.mbrgm_id as case_id, case when mbr_hist.meck is not null and mbr_hist.efdt is not null and mbr_hist.efdt…
venkat
  • 111
  • 1
  • 1
  • 11
-1
votes
3 answers

Want to use multiple aggregate function with snowflake pivot columns function

CREATE TABLE person (id INT, name STRING, date date, class INT, address STRING); INSERT INTO person VALUES (100, 'John', 30-1-2021, 1, 'Street 1'), (200, 'Mary', 20-1-2021, 1, 'Street 2'), (300, 'Mike', 21-1-2021, 3, 'Street 3'), …
Parth Desai
  • 1
-2
votes
0 answers

Need help to ingest data in azure databricks from kafka

I need help to ingest data in azure databricks sql warehouse database table from kafka in a batch job which needs to run every hour and only the new data in kafka should be synced in the databricks sql table. Please let me know how this can be…
Jay Dharamshi
  • 1
-2
votes
1 answer

SQL - Categorize Transactions by group

I want to add a column to my table called Group, based on the on the column Category. I have already filtered the table so its already dog/puppy or multi. Does anyone one know how to do…
sm98123
  • 1
  • 2
-2
votes
1 answer

SELECT/extract the specific part of string BUT EVERY ROW HAS DIFFERENT LENTH

Hei, I have a table LOG (Result - Picture) that shows how many meters a truck i warehouse treveled per day. I want to SUM meters per day after and before optimisation and export them as METERS BEFORE and METERS AFTER per DAY. When I excute my SQL…
Andrijana K
  • 1
  • 2
-3
votes
1 answer

Modifying SQL query to Data bricks SQL query

Rewriting Oracle SQL query to Data bricks SQL query. Can anyone please help me on below Oracle SQL query that is compatible in Data bricks. SELECT DISTINCT STT_O_ID AS ANLY_S_ID, OG AS OG_ID FROM (SELECT DISTINCT M.STT_O_ID, …
RatnakarRao M
  • 9
  • 4
1 2 3
…
23
24