Need to change column name in adls data files in delta format. Online research is referring to renaming columns in delta table. I created delta table on the top of existing delta file and did rename column - It is renamed only in hive metastore , but not in backend file in adls.
Let me know how to achieve this. fyi - code used to rename which does not rename backend file in adls
ALTER TABLE TelephoneCommunication_Scrubbed SET TBLPROPERTIES ('delta.columnMapping.mode' = 'name', 'delta.minReaderVersion' = '2','delta.minWriterVersion' = '5');
ALTER TABLE <Table name> RENAME COLUMN contactid TO <New column name>;
Update : I ended up reading and writing back with overwriteschema option in pyspark
