0

I'm trying to run polybase in SQL Server 2022.

When running EXEC sp_configure @configname = 'hadoop connectivity', @configvalue = 7;

I get the error

Msg 46934, Level 16, State 1, Procedure sp_configure, Line 177 [Batch Start Line 4] The option 'hadoop connectivity' cannot be enabled in this edition of SQL Server.

I'm running this command, as I am trying to enable Azure storage for Polybase. This worked in 2019.

I don't understand why I'm getting this error.

Both Polybase services are running and I can query external SQL Server tables.

Neil P
  • 2,920
  • 5
  • 33
  • 64
  • What edition of SQL Server are you using? What does `PRINT @@VERSION;` return? – Thom A Jan 04 '23 at 16:52
  • @Larnu Great question, forgot to mention this is on developer - Microsoft SQL Server 2022 (RTM) - 16.0.1000.6 (X64) Oct 8 2022 05:58:25 Copyright (C) 2022 Microsoft Corporation Developer Edition (64-bit) on Windows 10 Enterprise 10.0 (Build 19044: ) (Hypervisor) – Neil P Jan 04 '23 at 16:53
  • 1
    From the [documentation](https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/polybase-connectivity-configuration-transact-sql?view=sql-server-ver16#general-remarks): *"Starting in SQL Server 2022 (16.x), Hadoop is no longer supported in PolyBase."* – Thom A Jan 04 '23 at 16:53
  • 1
    Further reading: [Functionality replacement options for Big Data and SQL Server](https://learn.microsoft.com/en-us/sql/big-data-cluster/big-data-options?view=sql-server-ver16#functionality-replacement-options-for-big-data-and-sql-server) – Thom A Jan 04 '23 at 16:56

1 Answers1

1

Hadoop is no longer support in Polybase in SQL Server 2022 onwards, this is documented in several locations, such as PolyBase Connectivity Configuration (Transact-SQL) - General Remarks:

Starting in SQL Server 2022 (16.x), Hadoop is no longer supported in PolyBase.

And more verbosely Introducing data virtualization with PolyBase - PolyBase Connectors:

PolyBase connectors

The PolyBase feature provides connectivity to the following external data sources:

External data sources SQL Server 2016-2019 with PolyBase SQL Server 2022 (16.x) with PolyBase APS PDW Azure Synapse Analytics
Oracle, MongoDB, Teradata Read Read No No
Generic ODBC Read (Windows Only) Read (Windows Only) No No
Azure Storage Read/Write Read/Write Read/Write Read/Write
Hadoop Read/Write No Read/Write No
SQL Server Read Read No No
S3-compatible object storage No Read/Write No No
  • SQL Server 2022 (16.x) does not support Hadoop.
  • SQL Server 2016 (13.x) introduced PolyBase with support for connections to Hadoop and Azure Blob Storage.
  • SQL Server 2019 (15.x) introduced additional connectors, including SQL Server, Oracle, Teradata, and MongoDB.
  • SQL Server 2022 (16.x) introduced the S3-compatible storage connector.

(Both Emphasis mine)

As such, if you are migrating from SQL Server 2019 to SQL Server 2022 and need similar functionality you will need to use a different solution. If this is deal breaking (at least for now) then stay on 2019, as Microsoft "will continue to support the functionality to the end of life of those products" for Hadoop.

Unfortunately, I will admit, I don't have familiarity with Hadoop, or PolyBase, however, Microsoft also offer extensive documentation on alternatives in Big data options on the Microsoft SQL Server platform - Functionality replacement options for Big Data and SQL Server.

Thom A
  • 88,727
  • 11
  • 45
  • 75