Stack Overflow
Stack Overflow

Questions tagged [azure-synapse]

Questions about Azure Synapse Analytics, bringing together enterprise data warehousing and big data analytics.

Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless or provisioned resources—at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs.

2525 questions
0
votes
2 answers

Azure Synapse Pipeline not detecting Notebook

Azure Synapse Pipeline not detecting Notebooks. It was working well no and didnt make any changes. I cant even open notebook inside of pipeline like there setting and open sign + also open sign disappeared which is weird. Problem When i create new…
Frank Ruz
  • 1
0
votes
1 answer

pyspark createDataframe typeerror: structtype can not accept object 'id' in type

An API call is returning DICT type response similar to the output below: {'Account': {'id': 123, 'externalIdentifier': None, 'name': 'test acct', 'accountNumber': None, 'Rep': None, 'organizationId': 123, 'streetAddress': '123 Main…
Alen Giliana
  • 2,144
  • 3
  • 17
  • 30
0
votes
1 answer

Azure Data Factory2's pagination rule (for HTTP API)

The concerning data is updated daily through an HTTP API, and no metadata is available. It consists of +/- 28 columns and 10,000 rows. There is a pagination of 30 applied to the API. It retrieves JSON data and is converted to Parquet through…
NL_Died
  • 1
0
votes
1 answer

Read JSON data from CSV in Synapse Azure

I have a CSV with some columns, one of them with JSON content. Can I query this JSON column with special handling? An example below: ei My goal is to run a query (openrowset documentation) and get output similar to this. ID Name 0 Valarie…
Joel Carvalho
  • 1
  • 5
0
votes
1 answer

Spark to Synapse "truncate" not working as expected

I have a simple requirement to write a dataframe from spark (databricks) to a synapse dedicated pool table and keep refreshing (truncating) it on daily basis without dropping it. Documentation suggests to use truncate with overwrite mode but that…
irfan ijaz
  • 1
0
votes
1 answer

How do you filter data flow in azure synapse?

I have created a dataflow using row_number and I want to extract only the values where row_number = 1, but I can't get it to work. Please tell me how to do this. image #azure #synapse #dataflow
otaku
  • 1
  • 3
0
votes
1 answer

How to store ONNX Machine learning Model as a Hexadecimal string for Azure Synapse?

I've already created a Model.onnx file, how can I convert in to .hex format to make prediction by SQL Script in Azure Synapse? I read this document but still can't understand how to do it. Can anyone explain?
llgx 10
  • 1
  • 1
  • 3
0
votes
1 answer

Synapse Dedicated SQL Pool - Copy Into Failing With Odd error - Python

I'm getting an error when attempting to insert from a temp table into a table that exists in Synapse, here is the relevant code: def load_adls_data(self, schema: str, table: str, environment: str, filepath: str, columns: list) -> str: if…
kn0t
  • 303
  • 6
  • 13
0
votes
1 answer

Can I get pipeline parameters in Synapse Studio Notebook dynamically without predefining parameter names in parameter cell?

In the Pipeline that triggers the Notebook, I'm passing some base parameters in, as below. Right now, all the materials I read instructed me to declare variables inside the Notebook parameter cell with the same names, as below. Is there a way that…
xiexieni9527
  • 111
  • 7
0
votes
1 answer

Azure Synapse Analytics: Parquet Create TempView and change column values

I am hoping if someone can help here. Below is part of script our developer used in Azure Synapse to read and create TempView from parquet file. It reads parquet file "Client" and create a TempView: aa_client: aa_client =…
Mich
  • 19
  • 1
0
votes
2 answers

Create procedure COPY INTO statement with column names from external storage

I would like to use the COPY INTO statement to copy a table with it's column names from an external storage. Would like to achieve it using a generic procedure which can use for different tables. Here below you find a draft. The input parameters…
Baobab
  • 5
  • 3
0
votes
1 answer

ADF pipeline not able to read DECIMAL(36,0) value from Parquet file

We're using a copy activity to copy parquet file data into our managed instance SQL server. The source is using a SQL Serverless query to read the parquet files. There's a new column coming through that is bringing in large values and causing…
Jason Campbell
  • 146
  • 1
  • 10
0
votes
1 answer

Azure Synapse mapping dataflow - REST source dataset not paginating from ITGlue API

This is my first post here so forgive me if I'm in the wrong place. I'm running a mapping data flow in Azure Synapse to query ITGlue's REST API. It is only returning one page of results, vs all of them. Here's the endpoint I'm querying:…
samDTMSP
  • 23
  • 2
0
votes
0 answers

Synapse workspace deployment override defaultValue of trigger schedule

A trigger on our Synapse workspace needs different trigger times on different environments. During deployment the times are overriden by the default value that's part of the TemplateForWorkspace.sjon file in the workspace_publish branch. }, …
justsander
  • 91
  • 5
0
votes
1 answer

PowerBI user cannot access Built In Serverless DB on Azure Synapse

We have an issue with a particular user connecting to our Serverless SQL pool via PowerBI. I created an Azure Synapse workspace, which contains the built in serverless SQL pool which has some Views set up to pull data from some Parquet files. As the…
jamiemax
  • 179
  • 13
1 2 3
…
99 100