Uploading PIL image object to Amazon s3 python

Question

I want to get an image from the web and upload it to Amazon s3. While doing so, I would like to check the image dimensions. I have the following code in Python 3:

from PIL import Image
import requests

# Get response
response = requests.get(url, stream= True)

# Open image
im = Image.open(response.raw)

# Get size
size = im.size

# Upload image to s3
S3.Client.upload_fileobj(
    im, # This is what i am trying to upload
    AWS_BUCKET_NAME,
    key,
    ExtraArgs={
        'ACL': 'public-read'
    }
)

The problem is that the PIL image object does not support read. I get the following error when i try to upload the PIL Image object im.

ValueError: Fileobj must implement read

It works when I just try to upload the 'response.raw', but I need to get the image dimensions. How can I change the PIL image object to a file-like-object? Is there an easier way to get the dimensions while still being able to upload the image to s3?

So the question is; how do I upload an image to s3 after getting the dimensions of an image?

nbeuchat · Answer 1 · 2019-05-23T10:20:29.600

40

You need to use a file-like object but you should not call getValue() contrary to the accepted answer. Using the following snippet, you can then upload the image to S3 using in_mem_file when calling upload_fileobj:

from PIL import Image
import io

# Open image
pil_image = Image.open(response.raw)

# Save the image to an in-memory file
in_mem_file = io.BytesIO()
pil_image.save(in_mem_file, format=pil_image.format)
in_mem_file.seek(0)

# Upload image to s3
client_s3.upload_fileobj(
    in_mem_file, # This is what i am trying to upload
    AWS_BUCKET_NAME,
    key,
    ExtraArgs={
        'ACL': 'public-read'
    }
)

The .seek(0) part is needed to rewind the file-like object if you see that the uploaded file is 0kB.

edited May 23 '19 at 10:20

answered May 21 '19 at 15:42

nbeuchat

6,575
5
36
50

1

This is the accurate answer. Thank you for providing this. I was trying the accepted answer and was kind of lost on what I did wrong. This really helped. – hax Nov 27 '19 at 22:51
'ACL': 'public-read' - requires s3:PutObjectAcl permission. Also, after manipulations with the file, the pil_image.format data may be lost. So it's good to save it somewhere immediately after opening the file. – Serhii Kushchenko Nov 08 '21 at 16:59

score 20 · Accepted Answer · answered Oct 08 '17 at 16:01

Instead of calling read() to get the file contents back, you 'save' the file to either a real file object or an file like object in memory. Then call getValue() on it.

Here is an example function you could pass file content into, print out the height and width, then return the file data back in a format that an AWS clients put_object function will accept as the Body parameter.

from PIL import Image
import io

def modify_image(image, format):
    pil_image = Image.open(image)

    # Prints out (1280, 960) 
    print(pil_image.size)

    in_mem_file = io.BytesIO()

    # format here would be something like "JPEG". See below link for more info.
    pil_image.save(in_mem_file, format=format)
    return in_mem_file.getvalue()

There are also separate width and height attributes here: http://pillow.readthedocs.io/en/3.4.x/reference/Image.html#attributes

See more about the file formats here http://pillow.readthedocs.io/en/3.4.x/handbook/image-file-formats.html

Note: Example uses Python 3.6.1

Just a nag for the next guy: `format=format` should probably be `format=pil_image.format` — alphazwest, Mar 22 '19 at 21:03

amarynets · Answer 3 · 2017-09-13T19:05:57.440

0

You should use io.BufferIO

response = requests.get(url, stream= True)
f = io.BytesIO(response.content)
image = Image.open(f)

edited Sep 13 '17 at 19:05

answered Sep 13 '17 at 18:36

amarynets

1,765
10
27

This does not change anything. The image opens fine. The problem occurs when i want to upload the image to s3. – john Sep 13 '17 at 18:54
Did you try to load `f` into aws? Use `f` instead `im` – amarynets Sep 13 '17 at 18:56
Yes, I tried that. Well first of all, `response` does not have an attribute called `read`, I am assuming you mean `response.raw.read()`. When I try to do that I upload an empty file_obj, because `f` has been emptied, when it is opened with Image.open(). – john Sep 13 '17 at 19:02
How about trying `content` attr from a response? – amarynets Sep 13 '17 at 19:06
It still just uploads an empty picture. If i remove the `Image.open(f)` part all together it works fine. – john Sep 13 '17 at 19:15
This is plain useless, the requests doc states that `If you set stream to True when making a request, Requests cannot release the connection back to the pool unless you consume all the data (i.e. calling reponse.content) or call Response.close. This can lead to inefficiency with connections.` so by calling response.content you're defeating the purpose of asking for a stream in the first place as it means that you try reading the whole thing. – Samuel Prevost Apr 10 '19 at 09:34

score 0 · Answer 4 · answered Aug 02 '19 at 12:50

In case you are working with FileStorage from werkzeug.datastructures, having parsed an image using reqparse, then there is no need to convert the image to PIL.Image, you can use seek(0) on FileStorage itself.

Also do not forget to specify file's Content-type.

parser = reqparse.RequestParser()
parser.add_argument('image', help='image cannot be blank', type=FileStorage, 
location='files', required=True)

args = parser.parse_args()
image = args['image']

image.seek(0)
s3_client.upload_fileobj(image, self.BUCKET_NAME, filename, ExtraArgs={'ContentType': 'image/jpeg'})

Uploading PIL image object to Amazon s3 python

4 Answers4