U-SQL is a query language designed for Azure Data Lake. It provides a way to mingle SQL keywords with syntactic C# expressions, so that within a single script, a programmer can schematize the data from an unstructured source, use SQL to aggregate the data into the desired form, and then write the output to a file or table.
Questions tagged [u-sql]
771 questions
2
votes
1 answer
How to throw a error or raise exception in U-SQL?
What is the mechanism used to raise an error or exception in a U-Sql script? I have a scenario where am processing a CSV file, and if duplicates are found in it, then I need to abandon processing.
In SQL, I could do raiseerror, what it the…
sppc42
- 2,994
- 2
- 31
- 49
2
votes
2 answers
USQL call to read json array without a key
I have a nested json that I am trying to flatten in usql. I cannot share the data, but the structure is similar to this.
{
"userlist": [user1, user1],
"objects": {
"largeobjects": [object1, object2, object3]
"smallobjects":…
Cristian Iosub
- 157
- 12
2
votes
1 answer
U-SQL job to query multiple tables with dynamic names
Our challenge is the following one :
in an Azure SQL database, we have multiple tables with the following table names : table_num where num is just an integer. These tables are created dynamically so the number of tables can vary. (from table_1,…
ben
- 381
- 1
- 3
- 14
2
votes
1 answer
Splitting Columns in USQL
I am new to USQL and I am having a hard time splitting a column from the rest of my file. With my EXTRACTOR I declared 4 columns because my file is split into 4 pipes. However, I want to remove one of the columns I declared from the file. How do I…
Eddy42
- 35
- 4
2
votes
0 answers
ACL verification fails for Data Factory Service Principal, although it has rwx permissions
I have a U-SQL script that executes successfully from VS Code with my personal credentials, but fails when triggered from a Data Factory pipeline. My personal account has Owner rights on the Azure subscription. ADF uses Service Principal…
Lazer
- 501
- 4
- 14
2
votes
1 answer
U_SQL script with R
I found out that you can execute R within U-SQL. So i took a R-script from one of our data-scientists and build a U-SQL script based on this sample script.
The adapted script:
DECLARE @INPUT_DAT string =…
jbazelmans
- 283
- 1
- 6
- 16
2
votes
0 answers
How to use USQL database project that has credentials?
Since CREATE CREDENTIAL is deprecated, and the new project type outputs .usqldbpack to be run with PackageDeploymentTool.exe, how do we handle external SQL database sources and credentials?
Database don't exists yet, so can't run. PowerShell to…
SondreB
- 808
- 7
- 14
2
votes
1 answer
USQL Escape Quotes
I am new to Azure data lake analytics, I am trying to load a csv which is double quoted for sting and there are quotes inside a column on some random rows.
For example
ID, BookName
1, "Life of Pi"
2, "Story about "Mr X""
When I try loading, it…
Anbarasan Dhanushkodi
- 113
- 1
- 8
2
votes
1 answer
USQL Unit testing with ADL tools for VS 2017 - Error after upgrading to 2.3.4000.x
One of the team member after upgrading the ADL tools for VS to version 2.3.4000.x, getting the below error..
Error : (-1,-1) 'E_CSC_SYSTEM_INTERNAL: Internal error!
The ObjectManager found an invalid number of fixups.
This usually indicates a…
obulis
- 41
- 3
2
votes
1 answer
U-SQL Error - Change the identifier to use at least one lower case letter
I am fairly new to U-SQL and trying to run a U-SQL script in Azure Data Lake Analytics to process a parquet file using the Parquet extractor functionality. I am getting the below error and I don't find a way to get around it.
Error - Change the…
Satya Azure
- 459
- 7
- 22
2
votes
1 answer
USQL - SQL.ARRAY get length?
For the context, I get data from a sensor and it store in string like this
"axes" : "...,1,23,21,0,12,10,212,12,..."
the size may change depending on the machine sending the data... So my goal is to store it like a SQL.ARRAY and I want to get the…
jdc
- 384
- 4
- 15
2
votes
1 answer
MultiLevelJsonExtractor - Extract the desired level
I have a JSON document that looks as follows:
{
"Region": "Main",
"MarketLocations": [
{
"MarketName": "Central",
"MarketId": 1,
"SalesCategories": {
"Produce": [
{
"Type": "Apple",
…
Randy Minder
- 47,200
- 49
- 204
- 358
2
votes
2 answers
I can't extract the node text with a Xpath
I have a XML file (test.xml) like this one:
This is the name
20180207
…
lucazav
- 858
- 9
- 24
2
votes
1 answer
Extract files from single day- U SQL
I am facing issues with a U SQL script. I am trying to get files which were created on current day from a directory. the file name will have the date in yyyyMMdd format. But when i try to extract data instead of taking only one days files i am…
vignesh asokan
- 137
- 3
- 12
2
votes
1 answer
How to skip first n rows in U-SQL job?
I want to run a U-SQL job to load the data from .txt file in the SQL table on Azure Data Lake store. I already have created database, schema and table in Azure data lake analytics.
Data in txt file are tab-limited, and I need to skip 2 first row. I…
Nedzad G
- 1,007
- 1
- 10
- 21