Stack Overflow
Stack Overflow

Questions tagged [u-sql]

U-SQL is a query language designed for Azure Data Lake. It provides a way to mingle SQL keywords with syntactic C# expressions, so that within a single script, a programmer can schematize the data from an unstructured source, use SQL to aggregate the data into the desired form, and then write the output to a file or table.

771 questions
4
votes
1 answer

U-SQL in on premises environment

I've just read amazing article about U-SQL in msdn magazine. U-SQL is great tool to analyze big data in microsoft azure. However, it is possible to use it in non-cloud environment, e.g. in SQL Server hosted on premises?
Liero
  • 25,216
  • 29
  • 151
  • 297
4
votes
1 answer

ADF triggered ADL jobs failing with syntax error

I am trying to run a job that runs successfully from within Visual Studio. I'd like to run this in my ADF pipeline but the job fails with a syntax error. ERRORID: E_CSC_USER_SYNTAXERROR SEVERITY: Error COMPONENT: CSC SOURCE: USER MESSAGE: …
chi
  • 471
  • 3
  • 18
3
votes
1 answer

U-SQL: how to deal with schema changes?

My original script is something like this: @input = EXTRACT A string, B string, C string, year string, month string, day string, filename string FROM @folder + "/{year}/{month}/{day}/{filename}.csv" USING…
BurgerBurglar
  • 95
  • 7
3
votes
2 answers

Execute R inside U-SQL

I'm trying to use U-SQL and R to forecast, so i need to pass from U-SQL to R a list of values, and return forecast from R to U-SQL All examples i found uses a reducer, so will process 1 row…
Jorge Ribeiro
  • 1,128
  • 7
  • 17
3
votes
4 answers

U-sql referenced assembly built with 4.5.1? Whereas documented 4.5 is needed

It's documented that U-SQL uses .net 4.5 https://learn.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-u-sql-programmability-guide#use-assembly-versioning So to ensure that our own custom assemblies are built to that runtime. When…
Alex KeySmith
  • 16,657
  • 11
  • 74
  • 152
3
votes
1 answer

Connecting Power BI to S3 Bucket

Need some guidance as I am new to Power BI and Redshift .. My Raw JSON data is stored in Amazon S3 bucket in the form of .gz files (Each .gz file has multiple rows of JSON data) I wanted to connect Power BI to Amazon s3 Bucket. As of now based on my…
Bond
  • 101
  • 1
  • 3
  • 15
3
votes
1 answer

Using U-SQL to eliminate duplicate and null values in one specific column while keeping a 2nd column properly aligned

I am trying to use U-SQL to remove duplicate, null,'',and Nan cells in a specific column called "Function" of a csv file. I also want to keep the Product column correctly aligned with the Function column after the blank rows are removed. So i would…
Royale_w_cheese
  • 297
  • 2
  • 9
3
votes
1 answer

Data Lake Analytics U-SQL EXTRACT speed (Local vs Azure)

Been looking into using the Azure Data Lake Analytics functionality to try and manipulate some Gzip’d xml data I have stored within Azures Blob Storage but I’m running into an interesting issue. Essentially when using U-SQL locally to process 500…
user3034104
  • 69
  • 5
3
votes
1 answer

Access Azure Data Lake Analytics Tables from SQL Server Polybase

I need to export a multi terabyte dataset processed via Azure Data Lake Analytics(ADLA) onto a SQL Server database. Based on my research so far, I know that I can write the result of (ADLA) output to a Data Lake store or WASB using built-in…
user796246
  • 33
  • 2
3
votes
3 answers

Azure Data Lake Loop

Does Azure Data Lake Analytics and U-SQL support use While or For to Loop and create multiple outputs? I want output to multiple files using one USQL execution. This is what i want: Foreach @day in @days @dataToSave = SELECT day AS…
Jorge Ribeiro
  • 1,128
  • 7
  • 17
3
votes
2 answers

u-sql: filtering out empty// Null strings (microsoft academic graph)

I am new to u-sql of azure datalake analytics. I want to do what I think is a very simple operations but ran into trouble. Basically: I want to create a query which ignore empty string. using it in select works, but not in WHERE statement. Below…
user1043144
  • 2,680
  • 5
  • 29
  • 45
3
votes
2 answers

Azure Data Lake - HDInsight vs Data Warehouse

I'm in a position where we're reading from our Azure Data Lake using external tables in Azure Data Warehouse. This enables us to read from the data lake, using well known SQL. However, another option is using Data Lake Analytics, or some variation…
MMartin
  • 172
  • 2
  • 10
3
votes
3 answers

virtual file set column and rowset variable U-SQL

I'm having an issue with scheduling job in Data Factory. I'm trying to approach a scheduled job per hour which will execute the same script each hour with different condition. Consider I have a bunch of Avro Files spread in Azure Data Lake Store…
taffarel
  • 4,015
  • 4
  • 33
  • 61
3
votes
2 answers

USQL msbuild - composite build output

I'm following the instructions on this link https://blogs.msdn.microsoft.com/azuredatalake/2017/10/24/continuous-integration-made-easy-with-msbuild-support-for-u-sql-preview/ It states that After running MSBuild from command line or as a VSTS…
databash
  • 656
  • 6
  • 19
3
votes
1 answer

What does "rerun in upstream in pipeline"?

I am defining a pipeline in data factory, I had some errors that I correct. The first activity is calling an usql script to do some aggregation, I changed the script plenty of time but the error is…
Nelson Gomes Matias
  • 1,787
  • 5
  • 22
  • 35
1 2
3
…
51 52