3

We are using Azure data factory to copy data from Salesforce to Azure Data warehouse. The job is getting failed since some values are getting truncated .The source column in Salesforce is of type Long Text area(32000). What will be equivalent data type in Azure data warehouse to accommodate this data from Salesforce ?

venkatesh sivalingam
  • 263
  • 5
  • 18
  • 1
    Are you using the PolyBase or bulkinsert writer within ADF? – ckarst Mar 22 '17 at 18:43
  • Bulk insert within ADF. – venkatesh sivalingam Mar 22 '17 at 18:55
  • 1
    I have to question the wisdom of storing text this long in a data warehouse. It's not like you can aggregate it or put it in a dimension or `GROUP BY`. If you really want to do something with the text, like analytics, sentiment analysis, named entity recognition etc consider placing this data in Azure Data Lake and using the cognitive libraries as per the fantastic examples [here](https://docs.microsoft.com/en-us/azure/data-lake-analytics/data-lake-analytics-u-sql-cognitive). You could then store the summarised outputs in the warehouse, even report from them, which would make sense. – wBob Mar 23 '17 at 09:54
  • I have to agreed @wBob – Paul Andrew Mar 23 '17 at 12:31

1 Answers1

0

Simply answer. Don't store 32,000 char fields in a data warehouse! :-)

Paul Andrew
  • 3,233
  • 2
  • 17
  • 37