I have pandas dataframe and I am trying to find the best way to save dataframe data to a hive table. One way is to save to csv file and load it to hive table. Is there a better way to do this ?
Asked
Active
Viewed 7,641 times
2
-
Check https://stackoverflow.com/questions/36919825/pandas-dataframe-in-pyspark-to-hive – meW Dec 20 '18 at 04:24
-
check out odo, http://odo.pydata.org/en/latest/ – Suresh Mali Dec 20 '18 at 04:39
-
These files are really small I am trying not to use spark for this purpose. – Bob Dec 20 '18 at 19:13
-
@Bob did you figure this out. What did you end up doing? – Dipayan Apr 07 '19 at 02:13
-
Without using PySpark: https://stackoverflow.com/q/57590374/12029183 – Oct 09 '19 at 06:55