Stack Overflow
Stack Overflow

Questions tagged [azure-synapse]

Questions about Azure Synapse Analytics, bringing together enterprise data warehousing and big data analytics.

Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless or provisioned resources—at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs.

2525 questions
3
votes
1 answer

Read schema information from a parquet format file stored in azure data lake gen2

I have a parquet format table stored in stored in azure data lake gen2 which is directly connected to an external table in azure synapse. I am trying to formulate a logic in sql language which will read the schema of that parquet file table and…
user14755618
  • 41
  • 5
3
votes
0 answers

Pyspark dataframe error in Synapase notebook

I am using an Azure Synapse notebook in pyspark to write contents from an api call into a dataframe which is an array and later write it to a database. Code: from pyspark.sql.functions import explode from pyspark.sql import SparkSession from…
paone
  • 828
  • 8
  • 18
3
votes
1 answer

An expression cannot be used with default constraints. Specify only constants for default constraints in azure synapse sql

I want to add a default constraint using CREATE TABLE in AZURE SYNAPSE SQL, but I received the below error message instead. Also tried in alter statement but no luck. I have tried multiple ways but no luck. Could you please help me with the…
vikas madoori
  • 147
  • 1
  • 11
3
votes
1 answer

Cannot find the CREDENTIAL '', because it does not exist or you do not have permission

I created external tables using the steps specified here: https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/develop-storage-files-storage-access-control?tabs=shared-access-signature. Within Synapse workspace, I am able to access external…
Waqas
  • 6,812
  • 2
  • 33
  • 50
3
votes
1 answer

"Distribution Option" error in "CREATE TABLE table AS" in SQL Server

I have been facing an issue with using CREATE TABLE AS. I have made three tables in a database. I intend to join the tables based on policy number and create a new table out of the joined data. CREATE TABLE [dbo].[New_Customer_Segmentation_Data] AS…
Khawaja Abdul Ahad
  • 39
  • 1
  • 4
3
votes
2 answers

How to use FOR XML Path in Azure Synapse

Can you please help me will FOR XML PATH works in Azure Synapse. If not what will be the alternative. Thanks
user15488631
  • 85
  • 2
  • 10
3
votes
1 answer

Read only Delta between 2 versions of deltaLake

Is there a way to read only the delta between 2 versions of deltaLake? specifically in Synapse I have tried %%pyspark import delta from pyspark.sql.functions import col, asc df_delta = spark.read.format("delta") \ .option("readChangeFeed",…
user1324887
  • 632
  • 3
  • 11
  • 32
3
votes
2 answers

Cross Database Queries in Azure Synapse, Azure SQL Database, Azure Managed Instance and On Premise SQL Server

We are looking at options for moving our on premise SQL Server(s) to Azure and trying to understand whether we will be able to run cross database queries should we have data residing across multiple database technologies both in Azure ( specifically…
vavacious
  • 33
  • 1
  • 4
3
votes
0 answers

Azure Synapse Analytics Spark pool vnet integration solution

So I'm hoping to be able to move some ETL pipelines over from Azure Databricks over to Azure Synapse Analytics, but I'm running into some issues with a vnet integration. The case is as follows: We have a vnet that is peered to a VNG-vnet that…
Twc
  • 57
  • 1
  • 6
3
votes
1 answer

How to give Schema Level permission in Azure Data Warehouse (Synapse)?

In our Azure SQL Data Warehouse (Synapse), we have two schemas dbo and staging. I want to give schema level permission to my Azure Active Directory Group users only to 'dbo' schema. The expected behaviour I'm looking for is that after giving this…
mas
  • 339
  • 7
  • 22
3
votes
1 answer

SQL Data warehouse DDL parser - How to parse DW specific constructs

My use case is to accept Azure Synapse DW DDL as input, parse it and extract SQL statements from it. Additionally I can benefit from not only the base statements (viz, DECLARE, CREATE, ALTER, SELECT, ...) but also other DDL constructs like PK…
Deven Ahluwalia
  • 89
  • 9
3
votes
2 answers

How can we parameterise Azure Synapse Spark Jobs?

The Spark Jobs UI in Azure Synapse has the option to pass command line arguments to the underlying code, but there doesn't appear to be any option to parameterise these arguments. Similarly the Spark Job Definition activity in Azure Pipelines…
Steve Homer
  • 3,852
  • 2
  • 22
  • 41
3
votes
1 answer

OPENROWSET() for Synapse dedicated pools?

Any plans to support open row set for dedicated pools? Advantages: More (and faster) parser options over External File Format such as row delimiter Can auto-infer schema More convenient to define the file format directly
Anders Swanson
  • 3,637
  • 1
  • 18
  • 43
3
votes
0 answers

Azure Synapse Analytics - exception Running a Dataflow

Using the preview of Synapse Analytics Workspace and the relates Synapse Studio, I have created a Data Flow that simply loads parquet file from a Datalake gen2 store into a table inside a SQL pool . Running the pipeline that contains only suchData…
Maurizio Dapri
  • 31
  • 2
3
votes
2 answers

PySaprk- Perform Merge in Synapse using Databricks Spark

We are having a tricky situation while performing ACID operation using Databricks Spark . We want to perform UPSERT on a Azure Synapse table over a JDBC connection using PySpark . We are aware of Spark providing only 2 mode for writing data . APPEND…
HimanshuSPaul
  • 278
  • 1
  • 4
  • 19
1 2 3
…
99 100