Stack Overflow
Stack Overflow

Questions tagged [databricks-sql]

Questions about Databricks SQL

For questions about Databricks SQL - a serverless data warehouse on the Databricks Lakehouse Platform that lets you run all your SQL and BI applications at scale with improved performance, a unified governance model, open formats and APIs, and your tools of choice

357 questions
0
votes
0 answers

Date given "mmyy". How do I convert this into a date?

For example, 0519 is the date given. mmyy. How do I convert this into a date? Preferably mm-dd-yyyy. TIA!
cmarun88
  • 11
  • 1
0
votes
0 answers

How to set a filter in SQL that is based on a field but set at a future date?

I am trying filter based on a date range that should only go to 2023-01-01 (or first day of the next year) that corresponds to a field and it's date range (current and previous dates). My current filter in the where statement includes this but…
Dinho
  • 704
  • 4
  • 15
0
votes
0 answers

Spark SQL count of Distinct values with where

I want to see the number for each category by using distinct. But receiving only ones (1) for every category Can you please suggest how can I change my statement? At the moment I have following: %sql select catNum , count(catNum) as Number from…
Anna
  • 1
  • 1
  • 4
0
votes
1 answer

Databricks - Handling json string with keys containing special characters?

I have a JSON string containing keys that have $. In databricks notebook, using select prefixed with r to denote a raw-literal: SELECT r'{ "key1": "value1", "$version": 74501, "$metadata": { "$modified":…
alhazen
  • 1,907
  • 3
  • 22
  • 43
0
votes
0 answers

Databricks Error in SQL statement: AnalysisException: Cannot up cast documents from array

I am getting the following error when running a SQL query and do not understand what it means or what can be done to resolve it. Any recommendations? View DDL: CREATE VIEW myschema.table ( accountId, agreementType, capture_file_name, …
bda
  • 372
  • 1
  • 7
  • 22
0
votes
1 answer

Handling spaces in the abfss using COPY INTO with Azure Databricks

I am trying to use the COPY INTO statement in Databricks to ingest CSV files from Cloud Storage. The problem is that the folder name has a space in it /AP Posted/ and when I try to refer to the path the code execution raises the below error: Error…
Arnold Souza
  • 601
  • 5
  • 16
0
votes
1 answer

Can you sort barchart by values in databricks-sql dashboard?

Can you sort barchart in databricks sql dashboard? ( https://docs.databricks.com/sql/user/dashboards/index.html ) I'm trying to create a barchart with in databricks-dashboard-sql tool and sort it by its value (not alphabetically by its category). On…
Tom
  • 496
  • 8
  • 16
0
votes
1 answer

GETDATE() in Databricks SQL - it works but is undocumented

In SQL Databricks, I know I should use current_date() to get what is in T-SQL is GETDATE(). We're using cluster Runtime 10.5 and the getdate() function actually works, but the word isn't highlighted and when I looked in the documentation I couldn't…
Hila DG
  • 688
  • 4
  • 12
0
votes
1 answer

Getting Databricks error Error SparkUnsupportedOperationException: [INTERNAL_ERROR] Cannot generate code for expression: outer

In Databricks sql while executing SQL with NOT EXISTS operator (using correlated subquery) its not working. Getting Databricks error Error SparkUnsupportedOperationException: [INTERNAL_ERROR] Cannot generate code for expression: outer. Below is the…
Niranjan
  • 1
0
votes
0 answers

How do I perform UPDATE command in Databricks SQL?

I am migrating SQL commands to Azure Databricks SQL. While running the below Update command in databricks SQL: UPDATE three as t1 SET prod = WEB_TRFC_VISIT_SMRY.PRDCT_ID where exists ( (SELECT SVC_TAG_ID, max(supp_sys_ID) as PRDCT_ID FROM…
Aman Fatehpuria
  • 1
0
votes
2 answers

Databricks SQL Query Nested Json column which is stored as string

So, I have data in a column which looks like this: select additional_data,typeof(additional_data) as type from table . I am interested in querying the field appName. However, I can't seem to find a way to do this. I have tried: select…
Saugat Mukherjee
  • 778
  • 8
  • 32
0
votes
1 answer

How to run notebook inside another notebook in databricks?

How to pass the dynamic path to %run command in databricks because the function used in another notebook needs to be executed in the current notebook?
Venkatesh
  • 91
  • 1
  • 9
0
votes
0 answers

Databricks SQL Admin can't modify queries

I would like to understand why Admins in Workspaces are different than in Databricks SQL. I'll explain myself. In Databricks Data Science and Engineering: As an admin you can modify any notebooks in workspaces In Databricks SQL: As an admin, on…
Arthur Clerc-Gherardi
  • 495
  • 1
  • 3
  • 16
0
votes
1 answer

Databricks SQL CONCAT function adding characters where not requested?

I'm trying to use the CONCAT function to concatenate two columns in Databricks SQL. However, when I try to use it the CONCAT functions is prepending and appending .0 to the values in the second column. For example, my data looks like this: ColA ColB…
Yoddlenod
  • 41
  • 8
0
votes
1 answer

why in Databricks the last part of running takes a lot of time?

I am using Databricks to create an algorithm for big data. I am wondering why the last 1% of my running process takes a lot of time? I am writing the result in S3, the result for 111991 data (out of 116367) is done in 5 minutes and just for the last…
user15649753
  • 475
  • 2
  • 12
1 2 3
…
23 24