0

I'm trying to save a dataframe with Date as a column as parquet file. The Date series is in the type of object.

On one computer, there is no problems (pandas version is 1.4.2, fastparquet version is 0.7.2). However, when I tried it on another computer (with the same version of pandas and fastparquet), error was reported like this.

ValueError: Can't inter object conversion type:
0  2016-01-01
1  2016-02-01
2  2016-03-01

I did some research, it seems that the type of Date has to be converted from object to datetime. However, if so, why does it work on my previous computer. Are there any ways to fix this issue and keep Date as dtype? Thanks.

Harry
  • 331
  • 1
  • 4
  • 14
  • can you provide the code you use to save the data as parquet? It's possible that on one environment you have pyarrow installed and it preempts the use of fastparquet. – 0x26res May 27 '22 at 08:49
  • @0x26res thanks. sorry the code is quite long, have no way to provide. So pyarrow and fastparquet can affect each other. I'd better to install only one of them, right? – Harry May 27 '22 at 15:46
  • Or you can specify which engine you want to use when calling read_parquet with `engine="fastparquet"` or `engine="pyarrow"` – 0x26res May 29 '22 at 13:24

0 Answers0