0

I have a delta data source in dataflow. In order to connect to it, I need to use a parameterized linked service; however, I cannot find where I can address the values for the linked service parameters:
The parameters are highlighted in the the screenshot below:
enter image description here

And in dataflow, I cannot see any prompt to address the values of my parameters: enter image description here

Thanks :)

ARCrow
  • 1,360
  • 1
  • 10
  • 26
  • If the answer was helpful , You can [Accept it as an Answer](https://meta.stackexchange.com/questions/5234/how-does-accepting-an-answer-work), so that others who encounter the same issue can find this solution and fix their problem – KarthikBhyresh-MT Oct 27 '21 at 08:33
  • @KarthikBhyresh-MT Thanks for your response, I think a Data Lake non-parameterized linked service is a more straightforward solution than using Azure Delta Lake linked service. – ARCrow Nov 01 '21 at 20:51

1 Answers1

0

It so appears when you create a Delta Lake linked service you don't have a provision for parameters!

Here is what I think makes sense: Delta format is only available as an inline dataset. Because Delta lake is an open-source built-in storage layer from Spark which runs on top of an Azure Data Lake or Databricks and doesn't have a schema, currently ADF has no provision for parameterized linked service for this. And since these files can be stored on Azure Blob Storage or ADLS or Databricks, you only have these as supported linked service sources.

enter image description here

Note: In your snip you are accessing Azure Data Lake Storage Gen2 linked service and not Azure Databricks Delta Lake linked service (natively doesn't has a provision to provide parameters)

enter image description here

For example a SQL linked service supporting parameters, would have its dataset configuration properties request for the relevant parameters during connection as seen below, this seems to be missing, as an inline dataset.

enter image description here

enter image description here

But .....

enter image description here

So when you have a configuration like below, you can only provide default values to the parameters. And confirm working by using test connection.

enter image description here

OR when you set the source files:

enter image description here

Any attempts to reference a dataflow parameter to linked service parameter by editing the JSON content in the advanced section or in the dataflow code/script were reset, mostly indicating the method does not support.

I couldn't find a clear documentation on the same, you can consider logging as support request or an issue here for an official response or share a feedback here, similar one

KarthikBhyresh-MT
  • 4,560
  • 2
  • 5
  • 12
  • Hi :) Thanks for your response. I didn't understand how the relationship between data flow parameters and linked service parameters were established. For example, how does data flow know that p_DLurl of data flow maps to DLurl of the linked service? Thank you :) – ARCrow Oct 21 '21 at 13:10