0

What's generally accepted wisdom/tech solution for temp files on GAE using PHP, please?

I need to store and process an image (a few hundred K in size) using PHP that momentarily (e.g. < 1 second) needs to exist as a temp file somewhere before it's sent on elsewhere and the temp file can be deleted. My site will need to autoscale for potentially large numbers of users (using GAE as standard).

My idea was to attempt to store the temp file in memory (using tempnam() etc) and if that failed (e.g. mem full on that instance), immediately try and use some other storage instead on the fly. Question is what? Image has to be available as a file for CURL to access it (I think?) and send elsewhere, so Memcache is not an option (can't access the data as a file to pass into CURL - or can I?), but e.g. Cloud Storage is (via 'gs://[bucket-name]/...'). Thing is, if I've just written the file to storage, is it immediately available for reading? That's also a significant cost incurred...

Any help much appreciated!

Thanks, Alex

Alex Kerr
  • 956
  • 15
  • 44
  • GCS is probably your best option here. Given that the images are relative small, you shouldn't run out of memory in most cases. – Mars Jun 14 '15 at 18:26
  • The answer will depend on where the image comes from (for example, is it a POST / upload?). Also the kin dog image processing you are doing and where it is going. Can you provide more details? I have used all-in-memory image processing with pipes before, but I can't tell whether your use case allows for this. Some more details would be really useful. – Tom Jun 15 '15 at 05:42
  • Thanks all. Image is being received direct from the user's browser via a POST. It then gets manipulated in-memory on the server. It then needs to be written to disc for a moment so it can be read back in via a CURL function for POSTing to another site. If I could get the CURL operation to read direct from the GD image output then I could dispense with the writing to file bit altogether but as it's a 3rd party SDK I'm using to do the final POST I'm not sure if this is possible or not... – Alex Kerr Jun 19 '15 at 15:45
  • The issues with using Cloud Storage for temp files that only exist momentarily are: 1) What storage costs will I incur (e.g. file is stored for 1 second say and then deleted - what does Cloud Storage billing count that as - 1 second? 1 hour? 1 day?) and 2.) If I write the file to Cloud Storage, how soon before it's available again for reading? – Alex Kerr Jun 19 '15 at 15:46

1 Answers1

1

GCS provides a default application free bucket that is tied to your app as well. There is no quota for this bucket.

Using a default bucket

To use the default bucket:

Determine whether your app already has the default bucket created:

  • Visiting the App Engine Admin console and select your app.
  • Select Application Settings in the left navigation pane.
  • If your app has a default bucket, the Application Settings page will contain the label Google Cloud Storage Bucket, with the bucket name listed under it. The default bucket name is typically .appspot.com, where is your application ID.

If your app doesn't have a default bucket, create one:

  • Visit the App Engine Admin console and select your app.
  • Select Application Settings in the left navigation pane and scroll to the bottom of the page to Cloud Integration
  • Click Create and wait for the bucket to be created. The bucket is completely set up and ready for use.

ref: https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/activate#using_a_default_bucket

Josh J
  • 6,813
  • 3
  • 25
  • 47