8

Here in Google App Engines I got this code that would help fetch an HTML code of any web page by its URL:

from google.appengine.api import urlfetch
url = "http://www.google.com/"
result = urlfetch.fetch(url)
if result.status_code == 200:
doSomethingWithResult(result.content)

I don't understand one thing here (among many other things, in fact) why it is suggested in this code to import urlfecth from google.appengine.api ? Does Python not have this command onits own?

brilliant
  • 2,489
  • 8
  • 26
  • 28

2 Answers2

17

Python has libraries such as urllib and httplib for fetching URLs, but on App Engine, all requests must go through the custom urlfetch library. App Engine includes stubs for urllib and httplib that cause them to use urlfetch internally, but if you have a choice, using urlfetch directly is more efficient and flexible.

Nick Johnson
  • 100,655
  • 16
  • 128
  • 198
  • 1
    Thank you, Nick, for this explanation. – brilliant Dec 12 '09 at 16:17
  • Source? Is it because of security reasons that they don't allow you to make network calls without using URL Fetch? – irwinb Aug 18 '12 at 01:27
  • 5
    @irwinb Source: I worked on the App Engine team for over 4 years. The reason is a combination of security implications of unfettered socket access, and practical issues that arrive from your app running on a computer that doesn't directly speak to the outside world. – Nick Johnson Aug 22 '12 at 10:03
6

google.appengine.api is a library that contains Google's version of urlfetch class. Quoting from the manual:

The URL Fetch service uses Google's network infrastructure for efficiency and scaling purposes.

Python has url retrieval classes in its standard library too, but those whould not be able to use the infrastructure that is available inside App Engine.

In short google.appengine.api urlfetch is more powerful, but there is nothing blocking from you from using Pythons own urllib either, that too is described in the manual page I linked above.

Anti Veeranna
  • 11,485
  • 4
  • 42
  • 63
  • 2
    if you use httplib or urllib on appengine it also uses google's network infrastructure. iirc they were added as wrappers for urlfetch and should behave more or less as their python stdlib cousins :) – tosh Dec 12 '09 at 16:49
  • 2
    tosh makes a very valid point. if standard python urls also use same infrastructure the i see no point in using url fetch as it makes the app less portable when you wan to exit GAE. – mataal Mar 18 '10 at 19:57