9

I am thinking about using Google App Engine.It is going to be a huge website. In that case, what is your piece of advice using Google App Engine. I heard GAE has restrictions like we cannot store images or files more than 1MB limit(they are going to change this from what I read in the GAE roadmap),query is limited to 1000 results, and I am also going to se web2py with GAE. So I would like to know your comments.

Thanks

Wooble
  • 87,717
  • 12
  • 108
  • 131
felix
  • 11,304
  • 13
  • 69
  • 95
  • depends on your requirements... – jldupont Dec 14 '09 at 19:53
  • 2
    all the information is public... so you can do more than "hear about it"... you can "read about it" too ;-) – jldupont Dec 14 '09 at 19:54
  • Are you actually planning to ask a question? I have been using GAE for a number of projects and I really like it, but it's a completely different kettle of fish from using a LAMP stack. I would suggest playing around with it and getting a feel for it. – drxzcl Dec 14 '09 at 20:00
  • 1
    What exactly would you like to know about? It's difficult to answer this based on what you've said. – Nick Johnson Dec 14 '09 at 23:11
  • 50MB blobs are now supported: http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-released-including.html – hoju Dec 15 '09 at 06:46

7 Answers7

8

Having developed a smallish site with GAE, I have some thoughts

  • If you mean "huge" like "the next YouTube", then GAE might be a great fit, because of the previously mentioned scaling.

  • If you mean "huge" like "massively complex, with a whole slew of screens, models, and features", then GAE might not be a good fit. Things like unit testing are hard on GAE, and there's not a built-in structure for your app that you'd get with something like (famously) (Ruby on) Rails, or (Python powered) Turbogears.

ie: there is no staging environment: just your development copy of the system and production. This may or may not be a bad thing, depending on your situation.

Additionally, it depends on the other Python modules you intend to pull in: some Python modules just don't run on GAE (because you can't talk to hardware, or because there are just too many files in the package).

Hope this helps

RyanWilcox
  • 13,890
  • 1
  • 36
  • 60
  • 4
    Not entirely true about staging: You can deploy an alternate major version, test it, then set it live if it works. – Nick Johnson Dec 15 '09 at 09:20
  • 1
    Nick is right. If you are wondering, you can go to "versions" in the left hand pane of the app dashboard. There you can set which version you want "live". You can also find urls for the other version so you can test them before setting them loose on the world. – drxzcl Dec 15 '09 at 09:45
  • 1
    And unit testing is not too bad in my experience with nosegae (http://code.google.com/p/nose-gae) and webtest (http://pythonpaste.org/webtest). – Koen Bok Dec 15 '09 at 12:00
  • you can usually zip up the module, so it only counts as one file – John La Rooy Mar 06 '10 at 04:36
  • @RyanWilcox, I dont see how you can have A without having B – Pacerier May 29 '19 at 12:32
5

using web2py on Google App Engine is a great strategy. It lets you get up and running fast, and if you do outgrow the restrictions of GAE then you can move your web2py application elsewhere.

However, keeping this portability means you should stay away from the advanced parts of GAE (Task Queues, Transactions, ListProperty, etc).

hoju
  • 28,392
  • 37
  • 134
  • 178
  • Mind that you can use the task queue with web2py (and in fact it uses it internally to do a sort of keep-alive so that GAE caches web2py code and serves pages faster) and you can use use transactions to they extend that GAE supports it (using web2py DAL code into GAE run_in_transaction). Sometimes to use some DAL functionality you may have to use the DAL API vs the web2py API. – mdipierro Dec 15 '09 at 18:02
  • Good point about Task Queue. I was thinking of my own use case where I need to postprocess uploaded files. – hoju Dec 15 '09 at 23:34
3

The AppEngine uses BigTable as it's datastore backend. Don't try to write a traditional relational-database driven application. BigTable is much more well suited for use as a highly-scalable key-value store. Avoid joins if at all possible.

Dan Lorenc
  • 5,376
  • 1
  • 23
  • 34
2

I wouldn't worry about any of this. After having played with Google App Engine for a while now, I've found that it scales quite well for large data sets. If your data elements are large (i.e. photos), then you'll need to integrate with another service to handle them, but that's probably going to be true no matter what with data of that size. Also, I've found BigTable relatively easy to work with having come from a background entirely in relational databases. Finally, Django is a somewhat hidden, but awesome, "feature" of Google App Engine. If you've never used it, it's a really nice, elegant web framework that makes a lot of common tasks trivial (forms come to mind here).

Bialecki
  • 30,061
  • 36
  • 87
  • 109
1

Google has just released version 1.3.0 of the SDK with support with a new Blobstore API for storage of files up to 50MB. See the post "App Engine SDK 1.3.0 Released Including Support for Larger User Uploads".

Antonio
  • 11
  • 1
  • 1
    web2py team is already working on supporting this out of the box and under the hood for Field(...,type='blob'). – mdipierro Dec 15 '09 at 18:03
0

What about Google Wave? It's being built on appengine, and once live, real-time translatable chat reaches the corporate sector... I could see it hitting top 1000th... But then again, that's an internal project that gets to do special stuff other appengine apps can't.... Like hanging threads; I think... And whatever else Wave has under the hood...

Alyxandor
  • 1
-11

If you are planning on a 'huge' website, then don't use App Engine. Simple as that. The App Engine is not built to deliver the next top 1000th website.

Allow me to also ask what do you mean by 'huge', how many simultaneous users? Queries per second? DB load?

Daniel Goldberg
  • 19,908
  • 4
  • 21
  • 29
  • 3
    I'm curious what makes you say that. All of GAE is built to scale and scale and scale, at the cost of some additional complexity. The only problem I see is that with GAE's pricing model you will either pay through the nose for it, or have it exceed quotas really fast. Am I missing something? – drxzcl Dec 14 '09 at 20:04
  • 2
    I'd like to know more as well. It doesn't seem helpful to just say App Engine is not built to deliver the next top 1000th website without saying why especially when the quantification of 'huge' is unknown. – dtc Dec 14 '09 at 20:38
  • 11
    App Engine is built for exactly that. – Nick Johnson Dec 14 '09 at 23:12
  • Dan, your answer is almost as vauge as the question :) How many queries per second is too much for an App Engine app? – Peter Recore Dec 15 '09 at 04:52