Koalas dataframes (the Pandas API to Spark) has a to_orc method to save in the ORC format.
How to call it telling it save compressed using the ZLIB method?
Asked
Active
Viewed 16 times
0
-
based on the doc, you can provide spark properties as a dict. so, my guess is `to_orc('path/path/name', {'compression': 'zlib'})`. also, spark has officially launched [Pandas API on Spark](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html#pandas-api-on-spark). `koalas` is from databricks. – samkart Aug 25 '23 at 07:29
-
@samkart write it as an answer that I'll accept it. – neves Aug 25 '23 at 14:34