0

I am trying to load a lot of csv files from blob storage to Azure SQL Data Warehouse through Azure data factory. As I am dealing with massive number of rows, the desired approach is to use PolyBase to bulk loading the data. When I point the source to one single file, SQL DW PolyBase is displayed as true but when I point to all csv files, the SQL DW PolyBase is displayed as false. Does anyone have experienced this issue?

SQLmojoe
  • 1,924
  • 1
  • 11
  • 15
milad ahmadi
  • 485
  • 3
  • 7
  • 18
  • Suggest adding the ADF tag for this since it's really an ADF UI issue. From a pure polybase perspective, you can point to a single file or a folder with a bunch of files for each table you wish to load/query. The latter will load all files into the same table (or query). If you tried pointing to the folder (not all the files within) in ADF and that failed, it's either an ADF limitation or defect (assuming you did things correctly). – SQLmojoe Aug 22 '18 at 19:28
  • Are you pointing the source to the storage container where all the files are? are the files named in a sequential order? – Piroinno Aug 05 '19 at 15:48

1 Answers1

0

You could always change the allow polybase to true in UI.enter image description here Or specify this property in json:

                    "sink": {
                        "type": "SqlDWSink",
                        "allowPolyBase": true
                         }
Fang Liu
  • 2,325
  • 2
  • 13
  • 18