How to write Spark XML writing with order by?

Asked Jul 30 '21 at 14:26

Active Aug 03 '21 at 10:39

Viewed 123 times

am trying to write xml file from my dataframe like below

 myDf.orderBy("name")
.repartition(1).write
.format("com.databricks.spark.xml)
.option("rootTag","colname")
.option("rowTag","colname2")
.save("filename")

This is writing a file but not following the order I specify. Every-time it gives different random order. How to get ordering proper in spark XML writing

edited Aug 03 '21 at 10:39

Nikunj Kakadiya

2,689
2
20
35

asked Jul 30 '21 at 14:26

Karthik

Is the dataframe before you write it ordered ? You can check it doing ```myDf.orderBy("name").show()``` – Omegaspard Aug 02 '21 at 12:40
Yes that is also ordered properly. But still everytime final write is giving random order – Karthik Aug 02 '21 at 17:07

How to write Spark XML writing with order by?

0 Answers0