Using solutions from this question: How to read a list of parquet files from S3 as a pandas dataframe using pyarrow?, I am importing the following in order to copy data from an s3 bucket to a snowflake table:
from snowflake.connector.pandas_tools import write_pandas
When I try to use the method "write_pandas" I get the error:
TypeError: __init__() missing 1 required positional argument: 'backend'
Here's my code:
conn = snow.connect(
user="ll",
password="zzz",
account="account",
role="role",
warehouse="WAREHOUSE",
database="db",
schema="C")
s3 = s3fs.S3FileSystem(anon=False, key = 'AKI', secret = 'zzz')
pandas_dataframe = pq.ParquetDataset('s3://bucket/folder2/File.snappy.parquet',
filesystem=s3).read_pandas().to_pandas()
write_pandas(conn, pandas_dataframe, "DATATABLE")
What do I need to do to add this required argument? Or is there a different solution?