How to implement Loops in U-SQL

Question

Is is possible to implement Loops (while/for) in U-SQL without using C#. If no, can anyone share the c# syntax to implement loops in u-sql.

I am extracting files from a particular date to a date, but right now I am extracting this by writing file path manually.

DROP VIEW IF EXISTS dbo.ReadingConsolidated;  
CREATE VIEW IF NOT EXISTS dbo.ReadingConsolidated 
AS

EXTRACT     
        ControllerID int?,          
        sensorID int?,
        MeasureDate DateTime,
        Value float

FROM  
"adl://datalake.azuredatalakestore.net/2015/7/1/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/2/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/3/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/4/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/5/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/6/Reading.csv",
"adl://datalake.azuredatalakestore.net/2015/7/7/Reading.csv"

Note: these files are present in different folder.

Can above thing is possible using loop?

score 2 · Accepted Answer · answered Dec 08 '16 at 10:06

The proper way to do this is to use virtual columns, then rely on partition elimination so that only files matching predicates will actually be read (you can confirm that in the job graph).

CREATE VIEW IF NOT EXISTS dbo.ReadingConsolidated 
AS

EXTRACT     
        ControllerID int?,          
        ParameterID int?,
        MeasureDate DateTime,
        Value float,
        date DateTime
    FROM  
    "adl://datalake.azuredatalakestore.net/{date:yyyy}/{date:M}/{date:d}/Reading.csv";


@res =
    SELECT * FROM dbo.ReadingConsolidated
    WHERE date BETWEEN DateTime.Parse("2015/07/01") AND DateTime.Parse("2016/07/07");

How to implement Loops in U-SQL

1 Answers1

Linked