Stack Overflow
Stack Overflow

Questions tagged [polybase]

PolyBase enables SQL Server 2016 instance to process Transact-SQL queries that read data from Hadoop/Azure Storage. The same query can also access relational tables in your SQL Server. PolyBase enables the same query to also join the data from Hadoop and SQL Server. In SQL Server, an external table or external data source provides the connection to Hadoop.

179 questions
1
vote
1 answer

Parquet file generated using polybase does not contain column names

I have used the following query to create an external table in SQL 2016 with polybase. CREATE EXTERNAL TABLE dbo.SampleExternal ( DateId INT NULL, CalendarQuarter TINYINT NULL, FiscalQuarter TINYINT NULL) WITH…
Dev
  • 31
  • 3
1
vote
1 answer

How to create External table in Azure without the location path exist

Is there anyway to create External table in Azure SQL DWH even though the location path mentioned in external table statement doesn't exist. For eg:- location '/src/temp' doesn't exist still I want external table to be created. create external table…
Ritesh
  • 1,030
  • 2
  • 12
  • 28
1
vote
1 answer

Benchmark test for polybase with azure data lake

Have anyone performed benchmark test using polybase with adl, I want to know if I am having a data file which is having 4milion rows, will polybase be helpful in fetching those rows to the data warehouse. Can anyone post any articles where I Can…
The User
  • 47
  • 7
1
vote
0 answers

Can we have primary key - foreign key relation ship between normal tables and external polybase tables in one database?

Can we have both normal tables and external tables in one database with primary key - foreign key relation ship between them? The external table is backed by blob storage and the file format is parquet. If we can have relation what if some one…
Joy George Kunjikkuru
  • 1,495
  • 13
  • 27
1
vote
1 answer

Use unicode field terminator in polybase

While creating PolyBase external file format definition for external data stored in Azure blob storage, i am struggling to specify field terminator as a unicode character. The reason why I want to specify unicode character is because when I am…
avi
  • 13
  • 3
1
vote
1 answer

HdInsight Azure DW Polybase to Hive Table (ORC) with Date partition is failing

I am trying to create a polybase external table in Azure Data warehouse for the Hive table which is stored as ORC(HD insight Data lake ) and partitioned by date when i query the external table its failing in SQL server without any proper error…
Nithesh kulal
  • 11
  • 1
1
vote
2 answers

SQL Server 2016 polybase error using hadoop

I have a simple pipe delimited file (newfile.txt) on my hdfs and I have configured my polybase correctly. I am having a tough time to import this file into my SQL Server using Polybase. Here are the queries: At first an external file format is…
user3838498
  • 29
  • 1
  • 1
  • 5
1
vote
1 answer

Does External file format in polybase support Row terminator?

I am loading a BCPed data which has few rows that contains newline character in data itself. So when I try to import those data it's throwing an error. To Solve this issue I need to specify row terminator in external file format as \r\n. Does…
HimalayanNinja
  • 421
  • 3
  • 11
1
vote
1 answer

Using Polybase in Azure SQL Data warehouse to access Hadoop (Azure IaaS) HDFS files

I am trying to use Polybase in Azure SQLDW to access delimited files in HDFS in Cloudera Cluster (provisioned as IaaS in Azure) however I am running into the following error: Msg 105019, Level 16, State 1, Line 40 EXTERNAL TABLE access failed due…
rh979
  • 657
  • 1
  • 5
  • 13
1
vote
1 answer

Polybase in SQL Server 2016

Just wondering if Polybase functionality in SQL Server 2016 would allow to search non-relational database. We use TRIM for our document management system. Would upgrading to SQL Server 2016 allow us to then search the TRIM documents? Thanks
Sawar
  • 11
  • 2
0
votes
0 answers

'cannot change type of fields: "publicKey", delete the fields and re-create them' Polybase Error

I am trying to change the value of some fields in my Polybase App but when I click save the error pops up and doesn't allow me to change anything. I Googled for the solution but neither Google nor their docs show me how to alter a field value. Any…
Garen Johnson
  • 1
  • 2
0
votes
1 answer

Azure synapse dedicated SQL pools exports to ADLS storage account via polybase have .parq extensions instead of .parquet

Azure synapse dedicated SQL pools exports to ADLS storage account via polybase have .parq extensions instead of .parquet extensions. CREATE EXTERNAL DATA SOURCE [SomeExternalDataSourcename] WITH(TYPE=HADOOP,…
harin pagidimuntala
  • 1
0
votes
1 answer

Issue with Polybase-from synapse analytics DW on S3

I am trying to create an external data source from azure synapse DW to S3 compatible bucket/storage. But I am getting an error about the unsupported URI. As per the Microsoft documents, S3 is compatible from synapse analytics. After creating the…
naveen
  • 3
  • 2
0
votes
1 answer

How to resolve authorization error while writing to synapse dedicated pool table from Azure Databricks using Pyspark using databricks service principl

I am using a databricks service principal which has Storage Blob Data Contributor access on ADLS db_datareader, db_datawriter, db_ddladmin permission on synapse dedicated sql pool I am using below code snippet to write the data to synapse table: I…
azuresnowflake1
  • 135
  • 1
  • 10
0
votes
1 answer

Polybase CETAS to Azure blob storage

I'm trying to write data to azure blob storage from a SQL 2022 VM, using polybase. I'm using managed identity for authentication with blob storage, as the VM is in a different VNET to the storage, this is the only way to allow the traffic through…
Neil P
  • 2,920
  • 5
  • 33
  • 64
1 2 3
…
11 12