Stack Overflow
Stack Overflow

Questions tagged [azure-synapse]

Questions about Azure Synapse Analytics, bringing together enterprise data warehousing and big data analytics.

Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless or provisioned resources—at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs.

2525 questions
0
votes
2 answers

Email errors from all activities in ADF at once

In Azure Data Factory I created a pipeline with several copy data activities. As soon as one of them crashes I want that it continues to the next step and that it sends an e-mail with the error. The mail error action is correct. When I link it to…
AntonyJ
  • 29
  • 6
0
votes
0 answers

Submitting spark jobs from VSCode to Azure Synapse stop working

Extension 'Spark & Hive Tools' to submit job from Visual Studio Code stop working for me and my colleagues from yesterday. https://marketplace.visualstudio.com/items?itemName=mshdinsight.azure-hdinsight&ssr=false#overview it is very strange that it…
Alexander
  • 331
  • 1
  • 2
  • 11
0
votes
0 answers

Azure Synapse Analytics - Can a Spark Job Activity return values?

EDIT: I have raised the same question on Microsoft Learn Q&A, and there they told me there is no solution as far as December 2022. They opened an internal ticket to solve this directly. My present solution is to write the outputs in the Lake…
Dario Ant
  • 1
  • 2
0
votes
1 answer

How to structure schemas better in Synapse SQL dedicated pool

Using Azure Synapse , Dedicated SQL pool. How can I structure my tables underneath a button that represents the schema? This is a small issue that really makes a big impact when many tables and schemas will be used in the database, and users will…
Emile Van Der Heyde
  • 1
0
votes
2 answers

Executing an OPENROWSET on azure synapse result in a 'Resolving CosmosDB path has failed with error 'Secret is not base64 encoded.''

I am deploying to an azure synapse environment queries using sqlcmd to serverless pool. The environment contains an SQL database that my deploying account has access to. I am creating first the credentials to access a cosmosDB with : CREATE DATABASE…
fidou
  • 11
  • 3
0
votes
0 answers

Reading from cosmos and loading in DWH Synapse taking 15-20 mins even it is 0 rows

I have been loading the data from cosmos DB to DWH Synapse using Synapse pipelines. I am able to load the data but it takes more time. For e.g. we are loading the data incrementally , so if there are no updates then it would be 0 rows read from…
Shailendra Kad
  • 31
  • 1
  • 9
0
votes
1 answer

Upserting rows with Azure Synapse from Dynamics 365 using versionnumber

I am new to Synapse Data Flows and have not been able to achieve, what might be a simple Transaformation. In Dynamics I have a table with over 3.5 million rows, that i am bringing daily with a simple dataflow (source > sink) into a Dedicated SQL…
Bernardo
  • 38
  • 4
0
votes
2 answers

what is the meaning of WITH (jsonContent varchar(MAX)) AS [result] in this code on azure synapse analytics

I am not able to undertsand why we are writing with (doc nvarchar(max)) as rows JSON…
Anubhav
  • 273
  • 2
  • 8
0
votes
1 answer

Call stored procedure from ADF and sink to Azure SQL database

I'm struggling with calling the stored procedure (create in SSMS, Azure serverless pools) in ADF and sink to AZURE database. I have a copy data activity, my Source dataset is linked to my synapse analytics serverless pools: My Sink is connected to…
Greencolor
  • 501
  • 1
  • 5
  • 16
0
votes
2 answers

How to schedule exports from Azure Resource Graph and LogAnalytics to a Storage account?

I've created a KQL query for a LogAnalytics Workspace and one for the Azure Resource Graph and now try to get the results scheduled to Synapse Analytics or Azure DataLake Storage Gen2. But from Synapse side I can only create a connection to Data…
Gerrit
  • 2,515
  • 5
  • 38
  • 63
0
votes
1 answer

Multiple lines in Azure Synapse/ADF expression builder

I would like to have all lines showed inside of the expression builder. I want to avoid the following: This works in pipeline runs, but is hard to debug and check. I want the following, which is easy to read and debug: Unfortunately this leads to…
jbizco
  • 1
0
votes
1 answer

How to copy many csv files using Synapse Pipelines from an online source with the date in the file name?

There is this git repository publicly available. It's being refreshed daily. There are several csv files with the structure like "DA-01-12-2022", "DA-02-12-2022", "DA-03-12-2022" and so on. The date is in the file name. It's also in the githublink,…
LightningStack575
  • 29
  • 5
0
votes
2 answers

Rank the dates in a table for each month

I need to find the last three distinct loaddates for each month in various tables for reporting purposes. Example: If I have data from 2021 February to today: I need the three loaddates of Feb 2021, March 2021 and so on till. Dec 2022 So far, I'm…
LearneR
  • 2,351
  • 3
  • 26
  • 50
0
votes
0 answers

errorCode": "6000" - azure synapse running the pipeline

I have error when running a pipeline in azure synapse. If I execute the synapse notebook manually it works good (readind and writing). But when I call the name notebook from the ForEach activity inside the pipeline it fails to run. I have the…
Greencolor
  • 501
  • 1
  • 5
  • 16
0
votes
1 answer

Delete __HIVE_DEFAULT_PARTITION__ USING spark Notebook

Tried everything for a few hours to delete a record with a column partition value of __HIVE_DEFAULT_PARTITION__ within my delta lake table using a spark notebook. I figured it out and will post the answer. For the record my partition column is…
bmukes
  • 119
  • 2
  • 9
1 2 3
…
99 100