4

I have a simple task: in luigi, store pandas dataframe as a csv in dropbox, using dropbox-python sdk

Usually (with S3, for example) you can use StringIO as a file-like in-memory object. It also works great with pandas df.to_csv()

Unfortunately, dropbox sdk requires binary type, and I can't get how to translate StringIO into binary:

with io.StringIO() as f:
    DF.to_csv(f, index=None)
    self.client.files_upload(f, path=path, mode=wmode)

TypeError: expected request_binary as binary type, got <class '_io.StringIO'>

ByteIO won't work with the df.to_csv() ...

Philipp_Kats
  • 3,872
  • 3
  • 27
  • 44

1 Answers1

6

As seen in this answer, dropbox expects a bytes object, not a file, so converting to BytesIO will not help. However, the solution is simpler than that. All you need to do is encode your string as binary:

csv_str = DF.to_csv(index=None) # get string, rather than StringIO object
self.client.files_upload(csv_str.encode(), path=path, mode=wmode)
JohanL
  • 6,671
  • 1
  • 12
  • 26