I have a pyarrow.Table
object that I want to pass to PySpark (and save as a Spark table).
How can I convert pyarrow.Table
to pyspark.sql.DataFrame
?
The only way I can see it to convert it to pandas.DataFrame
, but aren't there some more direct and reliable ways?