1

I am struggling to understand something here and im sure the answer is simple....when I run this command in a Databrick note book:

(df.write
 .format("delta")
 .option("path", "/file/path/location")
 .saveAsTable("MyTable"))

It creates a delta table. Okay Great! but if I run the same command in azure synapse spark notebook....it creates a table... Does Synapse now support delta tables? According to this stack overflow post it does not.

So my question is...whats the difference?

Thanks in advance!

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
B_HOP
  • 27
  • 4

1 Answers1

2

Azure Synapse Analytics has a number of engines such as Spark and SQL. Synapse Spark pools support Delta Lake. Synapse Serverless SQL pools recently supports reading from Delta Lake. Synapse Dedicated SQL Pools do not support Delta Lake at this moment.

That other StackOverflow answer is out of date. It was only discussing Synapse Serverless SQL Pools. I added a comment asking him to update it.

GregGalloway
  • 11,355
  • 3
  • 16
  • 47
  • Is there a way to set a data retention policy for the logs and files of delta tables in serverless sql pools? – dragon_fury_ Sep 01 '22 at 12:13
  • @dragon_fury_ for delta table, you can use the 'vacuum' command: https://docs.delta.io/latest/delta-utility.html#remove-files-no-longer-referenced-by-a-delta-table – David Wu Nov 25 '22 at 15:32