Stack Overflow
Stack Overflow

Questions tagged [urlfetch]

Use in addition to tags Google App Engine / Google Apps Script. Urlfetch is a built-in library that is used to fetch external URLs. Unlike other data fetching modules/libraries, data is fetched synchronously blocking until the result is returned.

Urlfetch is a built-in library that is used in Google products such as Google Apps Script or Google App Engine to fetch external URLs. Unlike other data fetching modules/libraries, data is fetched synchronously blocking until the result is returned.

Useful links

  1. UrlFetchApp class reference;
  2. HTTPResponse class reference;
  3. OAuthConfig class migration reference;
  4. Request origin pool of IP addresses;
  5. urlfetch module for Python 2 App Engine API reference.

Related tags

826 questions
11
votes
1 answer

GoogleAppEngine urlfetch timeout exception

Has anyone got any experience with the following exception when using GAE urlfetch? DownloadError: ApplicationError: 2 timed out I'm trying to send a HTTP POST request. Like so: result =…
nafe
  • 379
  • 5
  • 16
10
votes
1 answer

How to fetch a wordpress admin page using google apps script?

I need to fetch a page inside my Wordpress blog admin area. The following script: function fetchAdminPage() { var url = "http://www.mydomain.invalid/wp/wp-admin/wp-login.php"; var options = { "method": "post", "payload": { …
tic
  • 4,009
  • 15
  • 45
  • 86
10
votes
2 answers

Invalid Argument with UrlFetch

I am trying to run a MongoLab (REST based access to MongoDB) query via Google Apps Script. The URL is generated from the logger is shown…
Kris
  • 103
  • 1
  • 4
9
votes
3 answers

UnicodeEncodeError when fetching url

I have this issue trying to get all the text nodes in an HTML document using lxml but I get an UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 8995: ordinal not in range(128). However, when I try to find out the type of…
r_31415
  • 8,752
  • 17
  • 74
  • 121
8
votes
2 answers

How can I suppress the urlfetch headers warning in Google App Engine?

Whenever I request an external URL using urlfetch on GAE I get the following warning: WARNING 2012-03-16 15:37:21,474 urlfetch_stub.py:428] Stripped prohibited headers from URLFetch request: ['Content-Length'] I understand why this is happening,…
alnorth29
  • 3,525
  • 2
  • 34
  • 50
8
votes
3 answers

App Engine: Alternatives to urlfetch? Seems very unreliable

I'm using urlfetch in my app and while everything works perfectly fine in the development environment, i'm finding urlfetch to be VERY unreliable when it's actually deployed. Sometimes it works as it should (retrieving data), but then a few minutes…
Don
  • 83
  • 6
8
votes
2 answers

Call to URLFetch failed with application error 5 for url x

Background: I am using google app engine and am having a weird bug in my site crawler. I have a backend that will automatically crawl a site every night. This is instigated by a task pushed to a pushQueue due to time limits in php. Problem: When I…
Grushton94
  • 603
  • 1
  • 7
  • 17
8
votes
4 answers

What Google AppsScript method is used to get the URL of a redirect?

'www.mysite.com/mySecretKey1' redirects to 'www.othersite.com/mySecretKey2' in G.AppsScript: var response = UrlFetchApp.fetch("https://www.mysite.com/mySecretKey1"); var headerString = response.getAllHeaders().toSource(); …
user3645994
  • 409
  • 6
  • 13
8
votes
2 answers

Why import urlfetch from Google App Engines?

Here in Google App Engines I got this code that would help fetch an HTML code of any web page by its URL: from google.appengine.api import urlfetch url = "http://www.google.com/" result = urlfetch.fetch(url) if result.status_code ==…
brilliant
  • 2,489
  • 8
  • 26
  • 28
7
votes
2 answers

Send HEAD request in Google App Script

I am developing a Google App Script to determine the size of a remote resource without downloading it. The code is as follows function getRemoteFileSize() { var params = { "method" : "head" }; var resp =…
Extreme Coders
  • 3,441
  • 2
  • 39
  • 55
7
votes
2 answers

google apps script ==> UrlFetchApp, method GET and cookie

I use the UrlFetchApp to send the user and pwd (method POST). After get the cookie, and use in other request (method GET). But this new request not working, I think that this cookie not has correct use in this new request. Can anyone help me? var…
user14272
  • 87
  • 1
  • 1
  • 5
7
votes
2 answers

UrlFetchApp muteHttpException doesn't work

How to handle urlfetch error? I'am trying muteHttpExceptions: true, but it doesn't work and the script breaks. function myFunction() { var result = UrlFetchApp.fetch("http://www.soccer-wallpapers.net/soccer_wallpaper/barcelona_fc_barcelona_360.jpg…
Meta Morfoself
  • 103
  • 1
  • 2
  • 9
6
votes
2 answers

URLFetch behind a Proxy Server on App Engine Production

Is there a way to specify a proxy server when using urlfetch on Google App Engine? Specifically, every time I make a call using urlfetch, I want GAE to go through a proxy server. I want to do this on production, not just dev. I want to use a proxy…
speedplane
  • 15,673
  • 16
  • 86
  • 138
6
votes
1 answer

How to fix the "Service invoked too many times for one day: urlfetch" error?

I am getting the following error in my Google sheet: Service invoked too many times for one day: urlfetch I know for a fact I am not making 100k calls, but I do have quite a few custom functions in my sheet. I tried to make a new sheet and…
kaitlynmm569
  • 1,605
  • 1
  • 6
  • 18
6
votes
2 answers

Asynchronous URLfetch when we don't care about the result? [Python]

In some code I'm writing for GAE I need to periodically perform a GET on a URL on another system, in essence 'pinging' it and I'm not terribly concerned if the request fails, times out or succeeds. As I basically want to 'fire and forget' and not…
John Carter
  • 6,752
  • 2
  • 32
  • 53
1
2
3
…
55 56