81

I am planning to create a web app that allows users to downgrade their visual studio project files. However, It seems Google App Engine accepts files uploading and flat file storing on the Google Server through db.TextProperty and db.BlobProperty.

I'll be glad anyone can provide code sample (both the client and the server side) on how this can be done.

Yeldar Kurmangaliyev
  • 33,467
  • 12
  • 59
  • 101
Graviton
  • 81,782
  • 146
  • 424
  • 602

11 Answers11

50

In fact, this question is answered in the App Egnine documentation. See an example on Uploading User Images.

HTML code, inside <form></form>:

<input type="file" name="img"/>

Python code:

class Guestbook(webapp.RequestHandler):
  def post(self):
    greeting = Greeting()
    if users.get_current_user():
      greeting.author = users.get_current_user()
    greeting.content = self.request.get("content")
    avatar = self.request.get("img")
    greeting.avatar = db.Blob(avatar)
    greeting.put()
    self.redirect('/')
sastanin
  • 40,473
  • 13
  • 103
  • 130
  • Don't like this approach cause you loose mime type info. – santiagobasulto Jan 16 '12 at 14:27
  • @santiagobasulto: Why don't you check it by ourself? – vietean Jan 20 '12 at 11:28
  • I don't want to check it. When you have to display an image you must provide mime type info (whether the image is JPG, GIF, etc) so, you must provide the content-type HTTP header. If you upload the image with the solution you provide then you won't know in the future the content type of the pic, ergo, you can't set the header, ergo, old-odd browsers will have problems to display the image (becouse of the absence of the content type) – santiagobasulto Jan 25 '12 at 15:24
  • 1
    If you don't check the mime type you are trusting the client, leaving you open to black hat attacks or misconfigured mime types in the client. If you are about to do that, you could just trust the file extension itself. – Nacho Coloma Dec 04 '12 at 15:58
45

Here is a complete, working file. I pulled the original from the Google site and modified it to make it slightly more real world.

A few things to notice:

  1. This code uses the BlobStore API
  2. The purpose of this line in the ServeHandler class is to "fix" the key so that it gets rid of any name mangling that may have occurred in the browser (I didn't observe any in Chrome)

    blob_key = str(urllib.unquote(blob_key))
    
  3. The "save_as" clause at the end of this is important. It will make sure that the file name does not get mangled when it is sent to your browser. Get rid of it to observe what happens.

    self.send_blob(blobstore.BlobInfo.get(blob_key), save_as=True)
    

Good Luck!

import os
import urllib

from google.appengine.ext import blobstore
from google.appengine.ext import webapp
from google.appengine.ext.webapp import blobstore_handlers
from google.appengine.ext.webapp import template
from google.appengine.ext.webapp.util import run_wsgi_app

class MainHandler(webapp.RequestHandler):
    def get(self):
        upload_url = blobstore.create_upload_url('/upload')
        self.response.out.write('<html><body>')
        self.response.out.write('<form action="%s" method="POST" enctype="multipart/form-data">' % upload_url)
        self.response.out.write("""Upload File: <input type="file" name="file"><br> <input type="submit" name="submit" value="Submit"> </form></body></html>""")

        for b in blobstore.BlobInfo.all():
            self.response.out.write('<li><a href="/serve/%s' % str(b.key()) + '">' + str(b.filename) + '</a>')

class UploadHandler(blobstore_handlers.BlobstoreUploadHandler):
    def post(self):
        upload_files = self.get_uploads('file')
        blob_info = upload_files[0]
        self.redirect('/')

class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler):
    def get(self, blob_key):
        blob_key = str(urllib.unquote(blob_key))
        if not blobstore.get(blob_key):
            self.error(404)
        else:
            self.send_blob(blobstore.BlobInfo.get(blob_key), save_as=True)

def main():
    application = webapp.WSGIApplication(
          [('/', MainHandler),
           ('/upload', UploadHandler),
           ('/serve/([^/]+)?', ServeHandler),
          ], debug=True)
    run_wsgi_app(application)

if __name__ == '__main__':
  main()
101010
  • 14,866
  • 30
  • 95
  • 172
  • Amazing that this works for everyone except for me... `self.__uploads` in `blobstore_handler` is `None` when I try this. – Tim Aug 08 '15 at 10:47
10

There is a thread in Google Groups about it:

Uploading Files

With a lot of useful code, that discussion helped me very much in uploading files.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
6

Google has released a service for storing large files. Have a look at blobstore API documentation. If your files are > 1MB, you should use it.

jbochi
  • 28,816
  • 16
  • 73
  • 90
6

I try it today, It works as following:

my sdk version is 1.3.x

html page:

<form enctype="multipart/form-data" action="/upload" method="post" > 
<input type="file" name="myfile" /> 
<input type="submit" /> 
</form> 

Server Code:

file_contents = self.request.POST.get('myfile').file.read() 
Bili
  • 79
  • 1
  • 2
3

If your still having a problem, check you are using enctype in the form tag

No:

<form encoding="multipart/form-data" action="/upload">

Yes:

<form enctype="multipart/form-data" action="/upload">
Joe Petrini
  • 489
  • 1
  • 3
  • 12
  • I was getting an encoding error before I implemented your answer – Jader Dias Jan 03 '10 at 20:44
  • 1
    A real annoying snag for me when I was doing this was not including "size" for the file input type. I was testing with Safari which apparently has a very short file length by default (?) and all I was getting in GAE for the file's contents was the file's name. Just a word of warning that might save someone a minor headache. – John Carter Nov 30 '10 at 00:44
1

Personally I found the tutorial described here useful when using the Java run time with GAE. For some reason, when I tried to upload a file using

<form action="/testservelet" method="get" enctype="multipart/form-data">
    <div>
        Myfile:<input type="file" name="file" size="50"/>
    </div>

    <div>
        <input type="submit" value="Upload file">
    </div>
</form>

I found that my HttpServlet class for some reason wouldn't accept the form with the 'enctype' attribute. Removing it works, however, this means I can't upload any files.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
  • 1
    It may be because you are using the get method, try setting it to post instead. I'm not certain if it'll work but it's worth trying. – slashnick Sep 09 '09 at 19:55
1

You can not store files as there is not a traditional file system. You can only store them in their own DataStore (in a field defined as a BlobProperty)

There is an example in the previous link:

class MyModel(db.Model):
  blob = db.BlobProperty()

obj = MyModel()
obj.blob = db.Blob( file_contents )
Guido
  • 46,642
  • 28
  • 120
  • 174
0

There is a way of using flat file system( Atleast in usage perspective)

There is this Google App Engine Virtual FileSystem project. that is implemented with the help of datastore and memcache APIs to emulate an ordinary filesystem. Using this library you can use in you project a similar filesystem access(read and write).

vivek_jonam
  • 3,237
  • 8
  • 32
  • 44
0

I have observed some strange behavior when uploading files on App Engine. When you submit the following form:

<form method="post" action="/upload" enctype="multipart/form-data">
    <input type="file" name="img" />
    ...
</form>

And then you extract the img from the request like this:

img_contents = self.request.get('img')

The img_contents variable is a str() in Google Chrome, but it's unicode in Firefox. And as you now, the db.Blob() constructor takes a string and will throw an error if you pass in a unicode string.

Does anyone know how this can be fixed?

Also, what I find absolutely strange is that when I copy and paste the Guestbook application (with avatars), it works perfectly. I do everything exactly the same way in my code, but it just won't work. I'm very close to pulling my hair out.

Honza Pokorny
  • 3,205
  • 5
  • 37
  • 43
  • 2
    :D The form said: mutlipart/form-data instead of multipart/form-data. Chrome is smart enough to correct the typo, Firefox isn't. – Honza Pokorny Aug 23 '10 at 12:30
0

There's no flat file storing in Google App Engine. Everything has to go in to the Datastore which is a bit like a relational database but not quite.

You could store the files as TextProperty or BlobProperty attributes.

There is a 1MB limit on DataStore entries which may or may not be a problem.

David Webb
  • 190,537
  • 57
  • 313
  • 299