1

I ran into an error when importing parquet file from Azure data lake to databricks. enter image description here

I tried other ways like importing parquet as Spark DataFrame successfully, but when I converted the Spark DF to Koalas DF, it gave the same error.

I also tried to import the data as Pandas DF successfully, but converting from Pandas DF to Koalas DF also failed.

enter image description here

I am new to Koalas. Can anyone shed some light on it. Thanks

Sajan
  • 1,247
  • 1
  • 5
  • 13
MiRe Y.
  • 57
  • 8

1 Answers1

1

I just found that a column was named like '._index_.' which caused the confusion for converting. I removed that column and it worked correctly.

MiRe Y.
  • 57
  • 8