0

I am running a website on google app engine written in python with jinja2. I have gotten memcached to work for most of my content from the database and I am fuzzy on how I can increase the efficiency of images served from the blobstore. I don't think it will be much different on GAE than any other framework but I wanted to mention it just in case.

Anyway are there any recommended methods for caching images or preventing them from eating up my read and write quotas?

clifgray
  • 4,313
  • 11
  • 67
  • 116

3 Answers3

1

Blobstore is fine.

Just make sure you set the HTTP cache headers in your url handler. This allows your files to be either cached by the browser (in which case you pay nothing) or App Engine's Edge Cache, where you'll pay for bandwidth but not blobstore accesses.

Be very careful with edge caching though. If you set an overly long expiry, users will never see an updated version. Often the solution to this is to change the url when you change the version.

dragonx
  • 14,963
  • 27
  • 44
  • an updated version of the image or the page itself? because my images and page change every few minutes when a user uploads or posts something – clifgray Sep 25 '12 at 17:11
  • Edge caching works by URL, so whatever the URL points to is cached. For you the best option is probably to set a short cache timeout. The other option is to have long cache timeouts, but change the URLs to your images as the images change. – dragonx Sep 25 '12 at 18:52
0

You can use google images api https://developers.google.com/appengine/docs/python/images/functions

What I usually do is on upload, i store the url created by the images.get_serving_url(blob_key). Not sure if its cheaper but on my dev server each call to get_serving_url creates a datastore write.

Faisal
  • 2,276
  • 15
  • 19
  • are writes more costly than reads? or were you just trying to save on writes specifically – clifgray Sep 25 '12 at 02:04
  • there's no datastore writes for get_serving_url in production. – Stuart Langley Sep 25 '12 at 03:58
  • @StuartLangley do you have any suggestions for caching images or is it even necessary? you usually know what you're talking about as far as GAE is concerned – clifgray Sep 25 '12 at 04:09
  • Interesting, thats why my other app does not consume as much, it just uses get_serving_url all the time. @Stuart do you know if get_serving_url is just as good as the cached url created? The method might not have a cache and has latency issue. Thanks – Faisal Sep 25 '12 at 15:25
  • Ohh what I meant was storing the result to datastore, was the cache. I actually just read the source, it does make an rpc call, that means it is better to just call it once and reuse it. – Faisal Sep 26 '12 at 03:30
0

My advice would be to use Google Cloud Storage for storing your images. It's better suited and recommended for serving static files. The good thing is that now you can use the same Images api for that:

Note: You can also serve images stored in Google Cloud Storage. To do this, you need to generate a Blob Key using the Blobstore API create_gs_key() function. You also need to set a default object ACL on the bucket being used that gives your app FULL_CONTROL permissions, so that the image service can add its own ACL to the objects. For information on ACLs and permissions, see the documentation for Google Cloud Storage.

PS. Another great feature I like here, is that you don't have to store different resolutions of your image if you need to serve them in different sizes. You can just add the parameters to the url which is returned by get_serving_url and that will do it. Also you only need to call get_serving_url once, store this url somewhere and use it whenever you need to serve the image. Plus you can reuse the same url for serving the same image in all different sizes.

URL Modifications:

=sXX To resize an image, append =sXX to the end of the image URL, where XX is an integer from 0–1600 representing the new image size in pixels. The maximum size is defined in IMG_SERVING_SIZES_LIMIT. The API resizes the image to the supplied value, applying the specified size to the image's longest dimension and preserving the original aspect ratio. For example, if you use =s32 to resize a 1200x1600 image, the resulting image is a 24x32. If that image were 1600x1200, the resized image would be 32x24 pixels.

=sXX-c To crop and resize an image, append =sXX-c to the end of the image URL, where XX is an integer from 0–1600 representing the new image size in pixels. The maximum size is defined in IMG_SERVING_SIZES_LIMIT. The API resizes the image to the supplied value, applying the specified size to the image's longest dimension and preserving the original aspect ratio. If the image is portrait, the API slices evenly from the top and bottom to make a square. If the image is landscape, the API slices evenly from the left and right to make a square. After cropping, the API resizes the image to the specified size.

husayt
  • 14,553
  • 8
  • 53
  • 81
  • 1
    there is no additional benefit using google cloud storage over the blobstore if you're going to serve the images using get_serving_url. – Stuart Langley Sep 25 '12 at 10:42
  • Using get_serving_url is almost free, because the images are not served by your own application. See this blog post: http://googleappengine.blogspot.nl/2010/08/multi-tenancy-support-high-performance_17.html – voscausa Sep 25 '12 at 12:04
  • it's not free - you're charged for bandwidth and a small amount of blob storage. – Stuart Langley Sep 25 '12 at 21:57
  • @StuartLangley There are a number of benefits you get with Cloud Storage: Free Hosting (at least to the end of the year), more flexibility with uploading files, CS is better suited for bigger files, it is a dedicated file storage engine with many additional features, shall you need them in future. Whereas with blobstore you would have to implement them manually shall you need them. To put it in perspective, CS allows you to do everything you can with blobstore and more. Thus it seems a better option. Plus, if you noticed Google is encouraging it by enabling blobstore api for CS. – husayt Sep 26 '12 at 17:48
  • yeh i know i wrote that API ;) – Stuart Langley Sep 26 '12 at 20:43
  • Classic, don't teach the teacher. Hello Stuart, good to know you are here. ;-)) Since you are here, maybe you can it clarify a little for us. It seems blobstorage going to be deprecated with CS, since I can't see any reason left to using them. Otherwise why would the apis be so aligned for takeover? – husayt Sep 27 '12 at 15:56