HTTP POST on app engine without waiting for result

Question

Currently spending a lot of instance hours just waiting for HTTP POSTs I send to Facebook to complete, even though I don't actually even look at the results of these POSTs (I am doing them to do page posts that users have scheduled through my app).

Even in asynchronous urlfetch, the docs say:

If any URL Fetch requests are pending when the request handler exits, the application server waits for all remaining requests to either return or reach their deadline before returning a response to the user.

Is there a way I can do a POST without waiting for the result?

This was asked once before Asynchronous URLfetch when we don't care about the result? [Python] but those answers are ancient, there might be some straightforward way now?

I'm currently considering putting them in some queue in memcached and then once a minute do all of them in parallel with asynchronous urlfetch, but maybe there's a better way. — Bemmu, Jul 07 '14 at 10:24
The instance hours aren't spent building the response, they are used doing the actual POSTing. So, they are necessary computing time for your process. You could send the task to the taskqueue, so the user doesn't have to wait, but the POST will still take CPU and write time. — GAEfan, Jul 07 '14 at 14:29
But what happens in the actual POSTing is that the request is formed, a connection opened, request contents written to the socket and then app engine starts just waiting for the response to come in. I would be fine not waiting for the response. — Bemmu, Jul 08 '14 at 01:05
I think there is some confusion. The time "waiting for the response to come in" is CPU time, where the server is doing the work you asked for. You pay for that time regardless of whether you are waiting for the response. The time to build the response and actually respond is miniscule. — GAEfan, Jul 08 '14 at 01:39
Ah, I must have been a bit ambiguous. I am the one sending the requests to Facebook. I am not serving the POSTs, I am sending them. — Bemmu, Jul 08 '14 at 23:52

score 1 · Answer 1 · answered Jul 09 '14 at 02:14

1

How about a urlfetch, with an extremely low deadline parameter?

https://developers.google.com/appengine/docs/python/urlfetch/fetchfunction?csw=1

answered Jul 09 '14 at 02:14

GAEfan

11,244
2
17
33

score 0 · Answer 2 · answered Jul 07 '14 at 16:01

I'd implement a taskqueue. This allows the request to finish once the item is inserted into the queue and the user won't have to wait until the process has finished. Write your request handler and remember to configure your queue with a queue.yaml file

Request handler

import webapp2
from google.appengine.api import taskqueue

class Request_Handler(webapp2.RequestHandler):
    def post(self):
        queue = taskqueue.Queue('queue-name')
        task = taskqueue.Task(url='/queue/url', params={'key': value})
        queue.add(task)
        self.response.out.write('success')

queue.yaml

queue:
- name: queue-name
  rate: 5/s
  bucket_size: 1
  max_concurrent_requests: 1

Actually this is already a task queue task, so there is no user waiting for it to complete. What I'm concerned about is if for example the POST takes 30 seconds to complete, if I'm not mistaken I am paying for the instance hours of those 30 seconds (total ~$180 / month). I don't want to wait because I don't need the result. Thanks! — Bemmu, Jul 08 '14 at 01:02

HTTP POST on app engine without waiting for result

2 Answers2