2

The external table creation using polybase was successful in local environment but it was unsuccessful in production where we use proxy servers for internet access. When tried to create external table in production environment I got following error "EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to HdfsBridge_IsDirExist: Error [com.microsoft.azure.storage.StorageException: An unknown failure occurred : Connection timed out: connect] occurred while accessing external file.'" I tried by enabling proxy settings using IE.Is there any way to enable proxy for SQL for creating external table which can establish connection to Azure blob.

Following are our requirement: - We need to run Polybase queries to create parquet files on Azure blob

  • We dont have direct internet connection on the DB server.

  • We need to use proxy to connect to internat externally.

  • We are able to create container through .Net Azure SDK after enbaling proxy inside the app.config file.

  • But we are not able to run the extenral table creation query from the SQL server and we are getting the below error.

"EXTERNAL TABLE access failed due to internal error: 'Java exception raised on call to HdfsBridge_IsDirExist: Error [com.microsoft.azure.storage.StorageException: An unknown failure occurred : Connection timed out: connect] occurred while accessing external file.' "

If this error is because of proxy issues, how can we give proxy for extenal table creation with azure reference.

eg: Sample external table creation is below -- EXTERNAL TABLE 

CREATE EXTERNAL TABLE dbo.SampleExternal (
    DateId INT NULL,
    CalendarQuarter TINYINT NULL,
    FiscalQuarter TINYINT NULL)
WITH (LOCATION='/SampleExternal.parquet',
    DATA_SOURCE=AzureStorage, 
    FILE_FORMAT=ParquetFile);

----- DATABASE SCOPED CREDENTIAL

CREATE DATABASE SCOPED CREDENTIAL AzureStorageCredential
WITH
    IDENTITY = 'user',
    SECRET = 'XXXXXXXXXXX=='
;

---DATA SOURCE 

CREATE EXTERNAL DATA SOURCE AzureStorage
WITH (
    TYPE = HADOOP,
    LOCATION = 'wasbs://XXContainer@XXStorage.blob.core.windows.net',
    CREDENTIAL = AzureStorageCredential
);
Dev
  • 31
  • 3
  • I'm not sure what proxy settings have got to do with it. Polybase (external tables) will access either Blob storage or Data Lake Storage "in the cloud". The most common reason for this error is that the directory does not exist. Can you please post your external table definition including data source and location? Obscure any confidential info you need to. – wBob Dec 11 '17 at 13:47
  • Thank you for your response.The directory exists in Azure Storage and extrenal table creation was successful from local development environment. I have updated the query with more details.Could you please have a look. – Dev Dec 12 '17 at 08:57
  • Have you tried putting the file in a folder? – wBob Dec 12 '17 at 21:14
  • No.This SampleExternal.parquet and corresponding folder will be generated after executing this external table script. – Dev Dec 13 '17 at 07:14
  • I suggest you try putting it in a folder. Did your other test version work like this? – wBob Dec 13 '17 at 09:22
  • It is working with one environment and that having internet connection. But in the testing environment we dont have any direct internet connection. we have to use proxy for connecting to internet. we were able to create and connect to azure blob container though .Net Azure SDK with defualt proxy in app.config. But when we run the polybase script it raises the above mentioned error. Not sure how to provide proxy while creating external tables. – Dev Dec 13 '17 at 11:40

0 Answers0