1

I have a dataframe with similar schema, I need to append the data into the AVRO file. I don't like to add the avro file into folder as a part. For your information, my AVRO file is not into the folder as a part. Can you please help me to solve the task.

T.SURESH ARUNACHALAM
  • 285
  • 2
  • 14

1 Answers1

0

You can write the data by using mode overwrite while writing the dataframe.

But the part file is created as spark is distributed processing and each executor spits out a files based on the amount of data

Shubham Jain
  • 5,327
  • 2
  • 15
  • 38