1

I'm working on a simple app that takes images optimizes them and saves them in cloud storage. I found an example that takes the file and uses PIL to optimize it. The code looks like this:

def inPlaceOptimizeImage(photo_blob):
        blob_key = photo_blob.key()
        new_blob_key = None

        img = Image.open(photo_blob.open())
        output = StringIO.StringIO()
        img.save(output,img.format, optimized=True,quality=90)
        opt_img = output.getvalue()
        output.close()

        # Create the file
        file_name = files.blobstore.create(mime_type=photo_blob.content_type)

        # Open the file and write to it
        with files.open(file_name, 'a') as f:
            f.write(opt_img)

        # Finalize the file. Do this before attempting to read it.
        files.finalize(file_name)

        # Get the file's blob key
        return files.blobstore.get_blob_key(file_name)

This works fine locally (although I don't know how well it's being optimized because when I run the uploaded image through something like http://www.jpegmini.com/ it gets reduced by 2.4x still). However when I deploy the app and try uploading images I frequently get 500 errors and these messages in the logs:

F 00:30:33.322 Exceeded soft private memory limit of 128 MB with 156 MB after servicing 7 requests total
W 00:30:33.322 While handling this request, the process that handled this request was found to be using too much memory and was terminated. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may have a memory leak in your application.

I have two questions:

  1. Is this even the best way to optimize and save images in cloud storage?
  2. How do I prevent these 500 errors from occurring?

Thanks in advance.

Nijin Narayanan
  • 2,269
  • 2
  • 27
  • 46
2748
  • 123
  • 2
  • 5

1 Answers1

1

The error you're experiencing is happening due to the memory limits of your Instance class.

What I would suggest you to do is to edit your .yaml file in order to configure your module, and specify your Instance class to be F2 or higher. In case you are not using modules, you should also add “module: default” at the beginning of your app.yaml file to let GAE know that this is your default module.

You can take a look to this article from the docs to see the different Instance classes available, and the way to easily configure them.

Another more basic workaround would be to limit the image size when uploading it, but you will eventually finish having a similar issue.

Regarding the previous matter and a way to optimize your images, you may want to take a look at the App Engine Images API that provides the ability to manipulate image data using a dedicated Images service. In your case, you might like the "I'm Feeling Lucky" transformation. By using this API you might not need to update your Instance class.

cjlallana
  • 636
  • 4
  • 12