U-SQL is a query language designed for Azure Data Lake. It provides a way to mingle SQL keywords with syntactic C# expressions, so that within a single script, a programmer can schematize the data from an unstructured source, use SQL to aggregate the data into the desired form, and then write the output to a file or table.
Questions tagged [u-sql]
771 questions
5
votes
1 answer
What is the maximum allowed size for String in U-SQL?
while processing a CSV file, I am getting an error about maximum string size. "String size exceeds the maximum allowed size".
Ahsan Abbas
- 155
- 7
5
votes
1 answer
Azure SQL Data Warehouse and PolyBase can't read CSV files generated using U-SQL and ADLA
I have a U-SQL script that generates a CSV file in Azure storage and an external table in Azure SQL Data Warehouse (SDW) using PolyBase to read that CSV file.
When the CSV file is generated using the U-SQL script, SDW is unable to read the data and…
Tiam
- 73
- 4
4
votes
1 answer
U-SQL Text Extractor Integer
I need to extract data from a file, see code below.
@rows =
EXTRACT booking_date string,
route string,
channel string,
pos string,
venta string,
flight_date string,
ancillary string,
paxs…
JSI
- 49
- 3
4
votes
1 answer
USQL Files vs. Managed Tables - how data is stored physically?
I am quite new to ADL and USQL. I went through quite a lot of documentation and presentations but I am afraid that I am still lacking many answers.
Simplifying a bit, I have a large set of data (with daily increments) but it contains information…
Jakub Krupa
- 127
- 9
4
votes
1 answer
Unable to install party package ( R) in ADLA
I am trying to install 'party' package in ADLA.
We have…
Arpit Sisodia
- 570
- 5
- 18
4
votes
2 answers
USQL Nesting TVFs and Queries is giving horrendous results
I 'think' that this problem is relating to the query optimization that Azure Data Lake Analytics does; but let's see...
I have 2 separate queries (TVFs) doing aggregations, and then a final Query to join the 2 together for final results.
So…
SimonB
- 962
- 1
- 14
- 36
4
votes
2 answers
Can i have any books about Azure Data Lake Internals?
I dont wanna use the ADL and ADLA as a black box. I need to understand how the gears rotate underhood to use it in an efficient way.
Where i can find an information that describe internals:
how U-SQL query is processed
how parallelism is…
churupaha
- 325
- 2
- 10
4
votes
1 answer
Is there any way to minimize U-SQL preparation time?
The preparation time on my U-SQL job is approximately 30 seconds. Is it possible to lower that at all?
My code is as follows:
USE DATABASE x;
USE SCHEMA y;
@results = SELECT RowKey
FROM y.tableName
WHERE…
Justin Borromeo
- 1,201
- 3
- 13
- 26
4
votes
2 answers
How to use subquery in USQL?
I am getting complilation error while using follwowing query in u-sql:
@CourseDataExcludingUpdatedCourse = SELECT * FROM @CourseData AS cd
WHERE cd.CourseID NOT IN (SELECT CourseID FROM @UpdatedCourseData);
It is not allowing me to use NOT IN…
Jai
- 416
- 6
- 20
4
votes
2 answers
U-SQL build error, equijoin have different types
I'm trying to create a USQL job and defined my columns from the CSVs they will be retrieved from, however I'm always having issues on the JOIN portion, because the columns I am matching are of a different type. This is weird because I have defined…
AnimaSola
- 7,146
- 14
- 43
- 62
4
votes
3 answers
Handling Files With Different Columns in USQL
I have a USQL script and CSV extractor to load my files. However some months the files may contain 4 columns and some months they may contain 5 columns.
If I setup up my extractor with a column list for either 4 or 5 fields I get an error about the…
Paul Andrew
- 3,233
- 2
- 17
- 37
4
votes
2 answers
Convert Rowset variables to scalar value
Is it possible to convert rowset variables to scalar value for eg.
@maxKnownId =
SELECT MAX(Id) AS maxID
FROM @PrevDayLog;
DECLARE @max int = @maxKnownId;
Pravin Dingore
- 127
- 1
- 8
4
votes
3 answers
Guid.NewGuid() always return same Guid for all rows
I need unique guid for every row i'm transforming from source.
below is sample script; code Guid.NewGuid() returns same always for all rows
@Person =
EXTRACT SourceId int,
AreaCode string,
AreaDetail …
Pravin Dingore
- 127
- 1
- 8
4
votes
3 answers
Unit testing for usql applier and scripts
I have a custom USql applier which extends the IApplier class.
[SqlUserDefinedApplier]
public class CsvApplier : IApplier
{
public CsvApplier()
{
//totalcount = count;
}
public override IEnumerable Apply(IRow input,…
Muhammad Umar Farooq
- 521
- 9
- 23
4
votes
6 answers
Implementing a Case Statement in U-SQL
Wondering if anyone has suggestions on implementing a case or IF statement with U-SQL. For example, how do i convert this:
SELECT
FirstName, LastName,
Salary, DOB,
CASE Gender
WHEN 'M' THEN 'Male'
WHEN 'F' THEN 'Female'
…
enufeffizy
- 121
- 2
- 5