Stack Overflow
Stack Overflow

Questions tagged [azure-synapse]

Questions about Azure Synapse Analytics, bringing together enterprise data warehousing and big data analytics.

Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless or provisioned resources—at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs.

2525 questions
2
votes
0 answers

Error when running queries against Synapse database

I am running a view against a Synapse linked server in SSMS. These are blob files. These links are contained in An Azure database. I get this error: Msg 46836, Level 16, State 1, Line 2 External table schema does not match actual schema from remote…
James Lawson
  • 21
  • 1
2
votes
1 answer

Can we use SQL scripts (Develop hub) during pipeline creation (Integrate hub) in azure synapse?

I want to use my SQL script (present under Develop hub) file inside a Pipeline (present under Integrate hub). Currently I do not see any Activities available solving this purpose. There is one Script activity under General section which only have a…
Gourav Dutta
  • 533
  • 4
  • 10
2
votes
0 answers

Using maven coordinates to load assemblies into synapse spark workspace

I would like to declaratively define the jars (package dependencies) that are used by the synapse workspace. Ideally I could do this by using maven coordinates. Is this possible? Currently the only way I've been able to add dependencies is by…
David Beavon
  • 1,141
  • 9
  • 16
2
votes
0 answers

I keep getting a "Failure to send request to Livy due to exception"

Trying to run a synapse pipeline but I keep getting the following error. "errorCode": "5000", "message": "Failed to send request to Livy due to exception=[Microsoft.Marlin.Common.Exception.LivyException].", "failureType": "UserError", …
Amzad
  • 21
  • 4
2
votes
2 answers

How to set the dependency between two trigger pipelines in azure synapse Analytics

I have a two trigger synapse pipelines one which is scheduled at 03 am cst , What I'm looking now is the Second pipeline should trigger after the completion of the first pipeline i.e after 03 am cst. Is there a way i can create this dependency in…
Never_Give_Up
  • 126
  • 1
  • 9
2
votes
1 answer

What is meant by an 'integration dataset' in Azure Synapse Analytics?

I'm new to Synapse. I am using 'Azure Synapse' and I have noticed that there is an option to import an 'integration dataset'. I'm not sure what exactly it means and how it differs from some of the other options for instance. I can't find anything…
user2361290
2
votes
1 answer

Synapse: Automated publishing for continuous integration and delivery without manual publishing ("The new CI/CD flow")

I'm trying to implement a fully automated ci/cd pipeline for my synapse workspace using azure devops piplines. I have a working version where everything except pushing the "Publish" Button inside the synapse UI after merging a feature branch into…
Robin M
  • 45
  • 5
2
votes
1 answer

Use SQLCMD Variables for User Name in DACPAC

I am trying automate the creation of users in a Synapse Dedicated SQL Pool. Building and integrating a DACPAC in an Azure DevOps pipeline works as described in the Microsoft Documentation. The problem is, that i want to specify the username at…
woozy
  • 33
  • 7
2
votes
1 answer

Azure Synapse Failure

Using the below code I am trying to copy the data from this code into a dedicated table. The data is coming from a Parquet that I need to alter before loading it into the table. I get the below error. Operation on target Move InvItemDist to DP…
James
  • 23
  • 5
2
votes
1 answer

Automatically deploying .whl artifacts from Azure DevOps feed to Synapse spark pools

I want to automatically deploy artifacts (custom python packages / .whl files) from the Azure DevOps artifact feed to a Synapse spark pool. Currently I have to manually: download the .whl file from the artifact feed upload the .whl file to the…
Cribber
  • 2,513
  • 2
  • 21
  • 60
2
votes
2 answers

Performance issue in Synapse serverless SQL pool while reading CSV stored in ADLS

I have enabled the Export to data lake feature in F&O D365 and created external table in Serverless SQL pool database in Synapse to read the CSV. It's working fine since 6 month however now I am facing performance issue due huge amount of data and…
Ajay
  • 247
  • 1
  • 5
  • 15
2
votes
1 answer

How to initiate tables lock for stored procedure in Azure Synapse Analytics?

Currently, I encountered an issue on Azure Synapse Analytics. I have a parent_cust_industry table which is full refresh - The table loads using stored procedure as below: CREATE PROCEDURE [test].[test_proc] AS BEGIN -- LOAD TYPE:…
Ken Masters
  • 239
  • 2
  • 17
2
votes
0 answers

D365 Azure Synapse Link Slow synchronisation

We are testing out Azure Synapse Link for Dataverse due to data export service being deprecated. We did some initial tests on low volume data, and were able to get the service up and running fairly easily. I've now tried setting it up for a larger…
Kuba
  • 21
  • 2
2
votes
1 answer

PySpark structured streaming with window get earliest & latest record based upon a timestamp value

I have a structured streaming process that reads from a deltalake. The data contains values that are increasing with time. Within each window, I would like to get the (difference between the) earliest and the latest record based upon the timestamp…
Kermit754
  • 343
  • 5
  • 14
2
votes
1 answer

Azure Synapse Spark LIVY_JOB_STATE_ERROR

i'm experimenting the following error when executing any cell in my notebook: LIVY_JOB_STATE_ERROR: Livy session has failed. Session state: Killed. Error code: LIVY_JOB_STATE_ERROR. [(my.synapse.spark.pool.name) WorkspaceType: CCID:<(hexcode)>]…
frammnm
  • 537
  • 1
  • 5
  • 17
1 2 3
…
99 100