I'm trying to use the vacuum command on a delta file located in Azure blob storage, which is accessed through Databricks. However, when I run the following code, the old versions of the file are not being removed:
path = '/mnt/containername/folder/myfile.delta'
from delta.tables import DeltaTable
deltaTable = DeltaTable.forPath(spark, path)
deltaTable.vacuum(24)
I've tried checking the history of the delta table after calling vacuum, and even tried overwriting the file first and then reading it and its history. However, every time I do this, I still see more file versions in the history output.
What should I do to actually clean old versions of a delta table file ?
