Stack Overflow
Stack Overflow

Questions tagged [azure-mapping-data-flow]

38 questions
0
votes
1 answer

How do you open a pdf file in python Juypterlab? I want to transform the data, but I can't get the file to open?

I need to convert data from a pdf file and transform it into a cleaned excel file. I need to take a dataset that is messy and clean it up. The issue is that it's a pdf file and I can't figure out what the code would be to open it. I'm unable to get…
Jenna
  • 1
  • 3
0
votes
1 answer

How to remove ambiguous date from the input tool?

I need to develop this alteryx condition at mapping dataflow level in adf. I tried by using filter tool to implement this implemented filter tool . but when i am performing union tool with this filter tool and with another tool. getting the error…
sony
  • 1
  • 1
0
votes
2 answers

Mapping Data Flow Azure

I am flattening a Json File to CSV file using mapping data flow in Azure data factory, the issue is that files is placed in storage container which is behind firewall. As it looks Only Auto Resolve integration runtime is supported in Mapping Data…
Anshul Dubey
  • 349
  • 4
  • 14
0
votes
1 answer

Which Spark version is being used by Mapping Data Flows?

I ran into some issues recently which lead me to believe that the Mapping Data Flows used in my Synapse Analytics Workspace are still running Spark 2.4, which has reached EOL. I tried to find out which Spark version is being used and how to change…
Alex AIT
  • 17,361
  • 3
  • 36
  • 73
0
votes
1 answer

Mapping data flow allows duplicate records when using UPSERT

Using Synapse pipelines and mapping data flow to process multiple daily files residing in ADLS which represent incremental inserts and updates for any given primary key column. Each daily physical file has ONLY one instance for any given primary…
Geekn
  • 2,650
  • 5
  • 40
  • 80
0
votes
1 answer

Azure Data Factory - replace expression in derived column transformation using Mapping Data Flow

I am trying to use Azure Data Factory for a Derived column transformation task one of my tasks is mentioned below, DESCRIPTION_TEXT : UNILEVER GROUP ##### GBR Remove trailing country code (only when it equals to country) and ##### if they exist My…
Rajkumar G
  • 3
  • 1
0
votes
2 answers

ADF Unpivot Dynamically With New Column

There is an Excel worksheet that I wanted to unpivot all the columns after "Currency Code" into rows, the number of columns need to be unpivot might vary, new columns might be added after "NetIUSD". Is there a way to dynamically unpivot this…
Candice
  • 31
  • 5
0
votes
0 answers

Dealing with dynamic variables in Mapping DataFlow in Data Factory

I am still struggeling with Mapping dataflow to make the flows more dynamic. When doing a toString(byName("static argument")) it seems to work nicely. Everything is good. But I would like to dynamically add argument and create an array like the one…
K N
  • 279
  • 5
  • 22
0
votes
1 answer

Filter rows in dynamic dataset based on json based rule set?

I need help with Mapping Data Flow in ADF. The problem is that I have a dynamic source with dynamic columns. The dataset column names are unknown at design time. I have a JSON document that describes rules like Column name and Value as regular…
K N
  • 279
  • 5
  • 22
0
votes
2 answers

How to provide LinkedService with a parameter value from Mapping Dataflow?

I am doing a SQL Azure Database as a datasource in a mapping dataflow. The parameter KeyVaultConnectionString should be dynamic. Se picture 1. My problem is that I am unable to fiugure out how to provide the value from mapping dataflow. I have tried…
K N
  • 279
  • 5
  • 22
0
votes
1 answer

ADF Data Flow flatten JSON to rows

IN ADF Data Flow how can I flatten JSON into rows rather than columns? { "header": [ { "main": { …
DC07
  • 293
  • 5
  • 18
0
votes
1 answer

Azure Data Factory inactive pipelines/resources cost

I want to know if there is a cost associated with pipelines, datasets, linked services, triggers and mapping data flows of Azure data factory when it is not in use. if there is a cost associated with it, is there any way I can deprovision or disable…
user961
  • 453
  • 6
  • 20
0
votes
1 answer

Azure mapping data flows - insert rows into two sink sequentially

I am implementing a data flow in which I need to insert rows into two different sink and both of the sink are Azure SQL database tables. The first sink i.e. the first Azure SQL database table has an auto generated ID column. After inserting values…
user961
  • 453
  • 6
  • 20
0
votes
1 answer

new record not visible in activity previews in mapping data flow

I have a mapping data flow with an Azure sql database source. For testing i have only 10 records in the source. If i insert a new record to the database table, then view preview of the source data in the mapping data flow, and click the refresh…
wilson_smyth
  • 1,202
  • 1
  • 14
  • 39
0
votes
1 answer

ADF mapping data flow only inserting, never updating

I have an ADF data flow that will only insert. It never updates rows. Below is a screenshot of the flow, and the Alter Row task that sets the insert/Update policies. data flow alter row task There is a source table and a destination table. There…
wilson_smyth
  • 1,202
  • 1
  • 14
  • 39
1
2
3