0

I have a requirement, where based on certain rules or conditions, the U-SQL script is generated. This is done via templating. I want some way to validate the generated U-SQL script, similar to the "compile script" feature in Visual Studio Code (for ADLA extension).

I found there are some references: Ref 1 Ref 2

Would appreciate if anyone can point out how to approach this.

The choice of programming language is Python 3

Joy
  • 92
  • 1
  • 9

2 Answers2

0

Azure Data Lake Tools supports multiple custom codes. You can develop U-SQL with Python, R, and C Sharp for Azure Data Lake Analytics in VS Code.

This article shows how to use Visual Studio Code (VSCode) to write Python, R and C# code behind with U-SQL and submit jobs to Azure Data Lake service.

For more details, refer Extend U-SQL scripts with Python code in Azure Data Lake Analytics

Hope this helps.

CHEEKATLAPRADEEP
  • 12,191
  • 1
  • 19
  • 42
0

I found out how to use it with Python(3). The OS is Windows. IDE used is VS Code.

Assuming all the extensions for Azure Data Lake ( and related extensions) already setup in VS Code, download the Local Run Service and Local Run Package. Set the SCOPE_CPP_SDK env variable (with correct path) in Windows (no need to set it up as system level env, you can choose to set it as user level env variable also).

import subprocess

Local_Run_SDK_Path = 'C:\Users\batman\AppData\Roaming\LocalRunService\LocalRunHelper.exe'
Path_to_Usql_file = 'C:\\MyProj\Usqls\Create_Tables.usql'
Path_to_Dataroot = 'C:\\DataRoot'

cmd = [Local_Run_SDK_Path, 'compile', '-Script', Path_to_Usql_file, '-DataRoot',Path_to_Dataroot]
proc = subprocess.run(cmd, stdout=subprocess.PIPE)

assert proc.returncode == 0 # if successful, the usql compilation should return 0 to proc return code

That should work.
Joy
  • 92
  • 1
  • 9