Stack Overflow
Stack Overflow

Questions tagged [polybase]

PolyBase enables SQL Server 2016 instance to process Transact-SQL queries that read data from Hadoop/Azure Storage. The same query can also access relational tables in your SQL Server. PolyBase enables the same query to also join the data from Hadoop and SQL Server. In SQL Server, an external table or external data source provides the connection to Hadoop.

179 questions
0
votes
2 answers

Choosing few columns in external table

I have a scenario wherein, I have a file(resides in ADLS) which contains 30 columns , but when i am creating external table.. i wanted to pick only 5 columns. My question is : is it possible to selectively choose the columns while creating external…
ravibhat
  • 811
  • 1
  • 7
  • 19
0
votes
2 answers

Sql Server 2019 Polybase feature issue

I'm trying to configure PolyBase on SQL Server 2019 to test some new features. I already installed this feature and performed all steps from below instruction: Install PolyBase on Windows I get an error notification when trying to configure…
bhuiokmnb7891
  • 3
  • 1
  • 5
0
votes
0 answers

Using Azure SQLDW Polybase for data ingestion from ADLS Gen 1 using vnet service endpoints

I am trying to use Polybase in Azure SQL Data Warehouse (SQLDW) to ingest data (persisted in Parquet format from a Hadoop cluster in a VNET) on Azure Data Lake Store (ADLS) Gen 1. The process is working fine but the throughput I am getting is quite…
rh979
  • 657
  • 1
  • 5
  • 13
0
votes
1 answer

issue in loading data with Azure data factory

I am trying to load a lot of csv files from blob storage to Azure SQL Data Warehouse through Azure data factory. As I am dealing with massive number of rows, the desired approach is to use PolyBase to bulk loading the data. When I point the source…
milad ahmadi
  • 485
  • 3
  • 7
  • 18
0
votes
1 answer

Connecting Azure Data Warehouse to Data Lake

From Azure Data Warehouse can we connect to Azure Catalog tables or Lake Store files as external tables? I want to be able to run SQL queries against the Azure Data Warehouse, but want to get the data from the Data Lake instead.
Vanamali
  • 47
  • 5
0
votes
1 answer

Polybase - is it possible to truncate over length fields

I've defined my external table definition, but I have one field that could theoretically contain a very long string that I dont care about. Rather than error out, is it possible to set polybase to truncate long fields that extend over the specified…
Neil P
  • 2,920
  • 5
  • 33
  • 64
0
votes
2 answers

ADF Onprem SQL Server to ADW Table copy issue

I am getting an error while performing the copy of a given on prem table to ADW, I have checked the data types and all, all looks good here but I dont know what breaking or whats been going on here, so that I can figure it out, Please have a look at…
John Wat
  • 11
  • 3
0
votes
1 answer

Pushdown in Polybase

I have the following scenario. A dimension table, e.g. PRODUCT is loaded into SQL Server 2016. A fact table, e.g. ORDER_ITEM is loaded into Hadoop. I want to run an aggregate query across PRODUCT and ORDER_ITEM, e.g. SELECT …
Uli Bethke
  • 782
  • 1
  • 7
  • 17
0
votes
1 answer

Can we use SQL Server 2016 Polybase without Azure?

I am trying to find a sample where i can see end to end installation of SQL Server 2016 polybase on local machine which connects to Hadoop not on Azure.
Divyesh
  • 1
0
votes
1 answer

Polybase performance issues

I am trying to connect my Hadoop clusters (Cloudera) from Polybase 2016. When I am querying Hadoop clusters from Polybase, the performance I am getting is very slow. It is in fact, 50 times slower than local SQL Server tables on same data size. Did…
Rituranjan Routray
  • 109
  • 1
  • 1
  • 6
0
votes
1 answer

Polybase to point to a file(s) locally on prem?

I have a unique situation where all my orc files are located on Azure on blob storage as compressed orc file. I need to load those files into on prem SQL DB. I can use polybase to download the data which I understand but it's taking extremely long…
HimalayanNinja
  • 421
  • 3
  • 11
0
votes
1 answer

PolyBase in SQL Data Warehouse : How to provide azure_storage_account_key as KEYURI in Database Scope Credential Creation

I am loading the data from Azure Blob storage container to SQL DW table through polybase. In The creation of Database scope credential I had to explicitly specify the azure storage key which I feel not secure. How I can replace it with a Key-URI…
ravi kiran
  • 371
  • 1
  • 5
  • 17
-1
votes
1 answer

Polybase : Querying the external table giving error but offending value shows nothing

I've compressed the text file in gzip format using powershell and uploaded into Azure blob . When i query the external table i'm getting the following error but offending value is nothing . Can any one tell me what is the issue and how can i find…
praveen pasumarthi
  • 3
  • 2
-4
votes
1 answer

Why can't Microsoft implement CETAS export for SQL Server 2016?

I wanted to export some data directly from SQL Server to Azure Blob storage in compressed orc format but there doesn't seem to be a way export like Azure SQL DW using CETAS Statement? I am wondering why Microsoft didn't include this feature in SQL…
HimalayanNinja
  • 421
  • 3
  • 11
1 2 3
…
11
12