google storage python API - uploading an StringIO object

Question

I succeeded in uplaoding a file to the google storage, but I would like to skip the creation of a file and use StringIO instead and create the file on the google storage directly.

Since I'm new to python, I just tried to use the standard method that I used for uploading the created file:

def cloud_upload(self, buffer, bucket, filename):
    buffer.seek(0)
    client = storage.Client()
    bucket = client.get_bucket(bucket)
    blob = bucket.blob(filename)
    blob.upload_from_filename(buffer)

But I get the error:

TypeError: expected string or buffer

But since I gave it the StringIO object, I don't know why this isn't working?

score 8 · Accepted Answer · answered Nov 16 '18 at 07:09

8

Hope this is still relevant. But you can try using blob.upload_from_string method

def cloud_upload(self, buffer, bucket, filename):
    client = storage.Client()
    bucket = client.get_bucket(bucket)
    blob = bucket.blob(filename)
    blob.upload_from_string(buffer.getvalue(),content_type='text/csv')

Do take note to modify the content_type parameter if you want to save the file as other file types. I placed 'text/csv' as an example. The default value is 'text/plain'.

answered Nov 16 '18 at 07:09

deedeeck28

347
2
8

Though this ans is correct, but once take a look in source code, `upload_from_string` simply convert string into `ByteIO` for `upload_from_filename` to upload. – MT-FreeHK Nov 16 '18 at 07:27

score 0 · Answer 2 · answered Aug 10 '20 at 07:56

0

Cause you are uploading from filename. But should upload from file object rather. Should be:

blob.upload_from_file(buffer)

answered Aug 10 '20 at 07:56

SirJ

173
2
17

score -1 · Answer 3 · answered Dec 27 '17 at 15:48

-1

I am not sure how are you invoking the cloud_upload function but you can simply use the upload_from_filename function to satisfy your need.

def upload_blob(self, bucket_name, source_file_name, destination_blob_name):
    """Uploads a file to the bucket."""
    storage_client = storage.Client()
    bucket = storage_client.get_bucket(bucket_name)
    blob = bucket.blob(destination_blob_name)

    blob.upload_from_filename(source_file_name)

    print('File {} uploaded to {}.'.format(
        source_file_name,
        destination_blob_name))

answered Dec 27 '17 at 15:48

patilnitin

1,151
10
14

well, I tried that, but I get TypeError: expected string or buffer...when I try to use buffer(StringIO object) instead of a file... – Tomislav Mikulin Dec 27 '17 at 16:38
Can you try with `blob.upload_from_filename(filename='/local/path.txt')` ? – patilnitin Jan 07 '18 at 08:32
1

yes I did something like that, but I wanted to upload the StringIO object directly - without making a file first and then uploading that! – Tomislav Mikulin Jan 07 '18 at 09:35

google storage python API - uploading an StringIO object

3 Answers3