1

Some context : When you use Azure Data Factory, you can create "Data Flows" which in turn contain graphical logic bricks, which themselves can transform data with an embedded code editor (the "Dataflow expression builder") that lets you execute scripts written in a native language called Azure Data Flows script (see picture).

enter image description here

Questions:

  1. Is that the real name of this language? Does it have a more common name? Is this language the twin brother of another, more widely adopted language?

  2. Is there an easy way of executing code snippets of it? Nowadays you can execute C# or many other languages directly in the web browser, with sites such as "Fiddle" and whatnot. Is there a place where I can test simple scripts of this scripting language without all the hassle of setting up Azure Data Flows, which are unbearably unreadable and unbearably slow?

Note: My specific use case is that I quickly want to test some regExp and text replacement, but even the Debug mode is excruciatingly slow, it takes several minutes to run the flow after each change.

jeancallisti
  • 1,046
  • 1
  • 11
  • 21

2 Answers2

1

The language is the Data Flow Expression language. Data Flows are executed as Spark Notebooks at run time, so the code is ultimately converted to Scala.

Since the conversion is handled internally, there is no way for you to interject custom code snippets. Synapse can more readily support custom notebooks than Data Factory.

Joel Cochran
  • 7,139
  • 2
  • 30
  • 43
0

If you're working at Pipeline level (not data flow level) then you may create a "set Variable" activity where you put the result of the expression. At least it lets you see the result of any text transformation or any other error-prone script.

jeancallisti
  • 1,046
  • 1
  • 11
  • 21