Stack Overflow
Stack Overflow

Questions tagged [u-sql]

U-SQL is a query language designed for Azure Data Lake. It provides a way to mingle SQL keywords with syntactic C# expressions, so that within a single script, a programmer can schematize the data from an unstructured source, use SQL to aggregate the data into the desired form, and then write the output to a file or table.

771 questions
3
votes
1 answer

Extracting data using U-SQL file set pattern when silent switch is true

I want to extract data from multiple file so I am using file set pattern that requires one virtual column. Because of some issues in my data, I also require silent switch other wise I am not able to process my data. It looks like, when I use virtual…
Jamil
  • 858
  • 11
  • 26
3
votes
2 answers

How to extract attribute values from XML element using XML Extractor in U-SQL

How can I extract attribute value from XML element using XML Extractor in U-SQL for my Azure data lake analytic job. Update: More details about the issue My XML file looks like this:
Jamil
  • 858
  • 11
  • 26
3
votes
3 answers

azure data lake u-sql pivot

I am loving Azure Data Lake but lack of documentation will probably slow down the adoption. I hope somebody out there have more experinnce on U-SQL than I do. Trying to derive from what's available under Microsoft.Analytics.Interfaces and via U-SQL…
chi
  • 471
  • 3
  • 18
2
votes
1 answer

Referencing ADL storage gen2 files from U-SQL

I have an ADL account set up with two storages: the regular ADLS gen1 storage set up as a default and a blob storage with "Hierarchical namespace" enabled which is connected to ADLS using storage key if that matters (no managed identities at this…
n0rd
  • 11,850
  • 5
  • 35
  • 56
2
votes
1 answer

How do we create a generic mapping dataflow in datafactory that will dynamically extract data from different tables with different schema?

I am trying to create a azure datafactory mapping dataflow that is generic for all tables. I am going to pass table name, the primary column for join purpose and other columns to be used in groupBy and aggregate functions as parameters to the…
chethan_raju
  • 91
  • 1
  • 8
2
votes
1 answer

"GetVersionedSdkPath: Unexpected layout" error trying to build U-SQL project

Two different computers, Windows 10 1803, brand new installs of VS2017 with v2.4.2000.0 of Azure Data Lake and Stream Tools. I try to build my ultra basic U-SQL project to run local and I get the following error... GetVersionedSdkPath: unexpected…
JKMajcen
  • 962
  • 1
  • 8
  • 7
2
votes
1 answer

VS update broke the microsoft.analytics.dll

I have a solution with two projects (C# and U-SQL) to run in the Azure Data Factory. I have referenced microsoft.analytics to use some analytics functions. After the VS update to 15.9.14 on July 9, 2019 the c# project was not building giving the…
Kiril1512
  • 3,231
  • 3
  • 16
  • 41
2
votes
1 answer

Unable to do conversion of datatype in U-sql

I am facing issue doing some transformations using U-sql one of the issue is while changing the Date format, Only when I skip the First rows(1) I am able to convert the date format. But I do need the column names so I cannot Skip the first row. Also…
Ravi
  • 45
  • 1
  • 4
2
votes
0 answers

ADLS Gen 1 poor performance vs General Purpose Storage Blob for ADLA U-SQL job

Acronyms ADLA : Azure Data Lake Analytics ADLS : Azure Data Lake Storage I've been trying to figure out my best option for storing the input data for an ADLA U-SQL job in either ADLS (gen 1) or a general purpose storage blob container, and have…
tones
  • 504
  • 1
  • 6
  • 17
2
votes
1 answer

Choosing between U-SQL and Spark / Databricks

I have experience in making ETL solutions based on Azure Data Factory and Azure Data Lake Analytics (U-SQL). But seems like Microsoft has started forcing Azure Databricks. Is U-SQL is dying? I have not seen any news about new features since…
Alex S
  • 103
  • 1
  • 2
  • 8
2
votes
2 answers

Calculating change in column over groups and extracting based on criteria

I am a beginner to coding in U-SQL/C#. I am stuck in a place during windowing/aggregation. My Data looks like Name Date OrderNo Type Balance one 2018-06-25T04:55:44.0020987Z 1 Drink 15 one…
Interested_Programmer
  • 322
  • 1
  • 3
  • 14
2
votes
1 answer

File Exists function in USQL

I am doing a simple test in USQL to check if a file exists and then perform Merge operations based on that. However, I cannot seem to get this right. Can you please help me with this and let me know where am I going wrong. DECLARE @InputFile1 string…
SDR
  • 361
  • 1
  • 4
  • 20
2
votes
1 answer

U-SQL + Pandas Merge_asof

I am working with Azure Data Lake Analytics for the first time and I am unsure how to merge 2 datasets like I would with pandas in python. I am merging two datasets that have different timestamps but I need to line them up if they are within a…
David Baumgarten
  • 19
  • 1
2
votes
1 answer

Write and Read an object with binaryformatting, u-sql

So I am currently attempting to create a couple of U-sql scripts one where a specific object is written to file, another script which reads the file and gets the object back. Write U-sql script : @bloomTest = EXTRACT date DateTime, …
user10362540
2
votes
1 answer

Update data in Azure Data Lake

I'm new to Azure Data Lake and big data in general and I apologize if my question seems stupid. I've been looking at ADL and ADLA to develop a cold path data storage. I've an Azure Stream Analytics query that outputs to Power Bi for real time…
MAK
  • 1,250
  • 21
  • 50
1 2 3
…
51 52