U-SQL is a query language designed for Azure Data Lake. It provides a way to mingle SQL keywords with syntactic C# expressions, so that within a single script, a programmer can schematize the data from an unstructured source, use SQL to aggregate the data into the desired form, and then write the output to a file or table.
Questions tagged [u-sql]
771 questions
2
votes
1 answer
USQL create user defined table type with user defined data type columns
I defined a User Defined Type as
namespace AddOns{
[SqlUserDefinedType(typeof(JsonObjectFormatter))]
public class JsonObject
{
public string Value {get;set;}
... // this is just a dummy representation
}
}
I want to…
frictionlesspulley
- 11,070
- 14
- 66
- 115
2
votes
1 answer
E_RUNTIME_USER_STRINGTOOBIG xml pasing
Trying to load XML through Azure Data Lake jobs I faced unbreakable issues with E_RUNTIME_USER_STRINGTOOBIG using Microsoft.Analytics.Samples.Formats
The thing is that XmlDomExtractor the only way to join (or rather apply) elements from inside the…
Vladimir Semashkin
- 1,270
- 1
- 10
- 21
2
votes
1 answer
U-SQL Error Extracting from TXT file
when running my extract, got this error:
Found invalid character-encoding for UTF-8 encoding in input. The
input file may contain corrupted data, or the specified input encoding
in the extractor does not match the actual file encoding. See the
…
Hari CR
- 43
- 3
2
votes
1 answer
Trouble using key phrase extractor and sentiment analysis with U-SQL through Azure data lake analytics on Azure data lake
I am trying to use microsoft's cognitive services with data lake and have run into a problem while trying to get key phrases and sentiment from the text in a column of a CSV file.
I have checked to make sure that the file is formatted correctly and…
Daniel
- 21
- 1
2
votes
1 answer
Fetch Paging Data using Azure Data Factory
I created a Pipeline in Azure Data Factory V2. It will copy the data from Rest API and save this data in the form of JSON file in Azure Data Lake. Then I transform that JSON file using U-SQL and Copy that data into another folder in .csv format. My…
Waqas Idrees
- 1,443
- 2
- 17
- 36
2
votes
1 answer
passing dynamic parameter to USQL script using data factory v2 USQL activity
USQL
I have multiple stored procedures defined in USQL database with the following signatures.
MyDatabase.dbo.DoThing01(@ticketNumber int)
MyDatabase.dbo.DoThing02(@ticketNumber int)
MyDatabase.dbo.DoThing03(@ticketNumber int)
These stored…
frictionlesspulley
- 11,070
- 14
- 66
- 115
2
votes
1 answer
How to handle carriage return, linefeed within a quoted string
Multiple source systems I want to process using Azure Data Lake contain a carriage return, linefeed within a column.
This causes Extract in ADLA to fail with the following error:
E_RUNTIME_USER_EXTRACT_UNEXPECTED_ROW_DELIMITER
Trying to find a…
RB84
- 331
- 4
- 15
2
votes
2 answers
auto_increment in U-SQL
I am trying to form a new table that contains unique user_id's from existing one. Is it possible to add auto_increment primary key in U-SQL like we can add in MySQL?
rmarathe
- 23
- 3
2
votes
1 answer
U-SQL equivalent of HASHBYTES to get hash of complete row
We have to store all the changes in a data row. I was looking for in build function like SQL Server to calculate HASHBYTES and time-stamp to find the changes and get the latest one as well.
Any pointer will be highly helpful.
Arnay
- 61
- 4
2
votes
3 answers
U-SQL splitting a column into two, delimited by "-"
I am trying to split a column into two by using U-SQL in data lake analytics. In SQL could do it like this:
,CASE WHEN [Total] like '%-%'
THEN TRIM(LEFT([Total],CHARINDEX('-',[Total]) - 1)) END AS [TotalLeft]
…
Richard
- 35
- 5
2
votes
1 answer
Unable to perform JSONPath conditional selection in USQL Job using json.net library
I have the following JSON file :
Created with the following source code :
{ "Person": { "PersonType": "BC", "PersonIdentification": { "IdentificationIdType": "INFOWARE_ID", "IdentificationId": "1009" }, "MatriculeRH": { "MatriculeIdType": null,…
saad0n87
- 67
- 6
2
votes
1 answer
VS2017 Data Lake tools cannot submit U-SQL script to Azure
Using Visual Studio 2017 with Data Lake Tools v. 2.3.1000.1, I am unable to submit a U-SQL job directly to Azure. I only have the option to submit locally:
This is the case even though I am connected to Azure through the "Server Explorer" tool…
Dan
- 10,480
- 23
- 49
2
votes
2 answers
Best way to combine spatial data
I have a question regarding spatial data and u-sql. I have a input with polygons and input with points and i want to combine these so that I know in what polygon the point belongs to.
In T sql I would do a left outer join with something like…
Tysklind-MSFT
- 87
- 6
2
votes
1 answer
Partition By & Clustered & Distributed By in USql - Need to know their meaning and when to use them
I can see that while creating table in USQL we can use Partition By & Clustered & Distributed By clauses.
As per my understanding partition will store data of same key (on which we have partition) together or closer (may be in same structured…
Mahesh Dounde
- 91
- 12
2
votes
1 answer
Azure / U-SQL - convert string to float
I'm having some issues while loading files in Azure data lake / U-SQL. I think there might be an issue with some of the values in one of the columns (I can load them with the column as a string but not as a float) so I'd like to load them as a…
screechOwl
- 27,310
- 61
- 158
- 267