1

I have upload the parquet data format file into databricks file System. Now I want to to store this parquet file data into Delta Table (Delta Format). But unable to do. My code is:

spark.read.option("inferschema",true).option("header",true).csv("/FileStore/tables/ExecutiveSummary.txt").
write.format("delta").mode("overwrite").save("/FileStore/tables/delta_train/")

It show me syntax error. What is the issue in this ?

I am using databricks community edition

Alex Ott
  • 80,552
  • 8
  • 87
  • 132
Learner1122
  • 15
  • 5

1 Answers1

0

You can't simply split code into two lines. Your first line is finishing with . character - it's incorrect in python. You need to use either this variant (with \ as continuation marker, and . put to the write word):

spark.read.option("inferschema",true).option("header",true).csv("/FileStore/tables/ExecutiveSummary.txt") \
  .write.format("delta").mode("overwrite").save("/FileStore/tables/delta_train/")

or this variant, with () around your expression, but still you need to put . in correct place:

(spark.read.option("inferschema",true).option("header",true).csv("/FileStore/tables/ExecutiveSummary.txt")
  .write.format("delta").mode("overwrite").save("/FileStore/tables/delta_train/"))
Alex Ott
  • 80,552
  • 8
  • 87
  • 132
  • any clue on this issue https://stackoverflow.com/questions/74035832/exception-occured-while-writing-delta-format-in-aws-s3 ? – Shasu Oct 12 '22 at 02:26