0

I'm trying task queues for the first time and while I think I have everything set up correctly, I am getting an error.

First, here is my queue.yaml file:

total_storage_limit: 500M
queue:
- name: loader
  rate: 1/s
  bucket_size: 1

Second, here is my code to call my task queue (it's called load.py). It needs to run every day, so I have this particular script run as a cron job. 

for file in archiveList:
        taskqueue.add(queue_name='loader',url='/tasks/loadworker',params = {'ID':file[:-4],'XML':str(file)})

My loadworker.py file is basically this:

class MainPage(webapp2.RequestHandler):
    def post(self):
        ID = self.request.get('ID')
        XML = self.request.get('XML')
        tmp = trialDatabase.get_or_insert(ID)
        #REST OF CODE GOES BELOW

Here is the error I am seeing:

WARNING  2014-04-11 15:24:41,156 taskqueue_stub.py:1974] Task task936 failed to execute. This task will retry in 0.400 seconds
INFO     2014-04-11 15:24:41,156 module.py:627] loadandprocess: "POST /tasks/loadworker HTTP/1.1" 404 -

Off the top of my head, I'm wondering: do the task queue "caller" and the task queue "worker" need to be in the same module? I have my task queue worker in my default app.yaml file, and my loader is in the loadandprocess.yaml file. How does it work for usage, like...would each task worker call a B4 instance class as specified in my loadandprocess.yaml file?

Thanks!

Edit:

Here is the relevant part of app.yaml:

- url: /tasks/loadworker
  script: loadworker.application
  login: admin
user3058197
  • 1,052
  • 1
  • 9
  • 21
  • 1
    Do you have a handler in your app.yaml file for /tasks/loadworker pointing to loadworker.py ? – IanGSY Apr 11 '14 at 15:53
  • Yes, see edited part above – user3058197 Apr 11 '14 at 15:54
  • Do you have something like __application = webapp2.WSGIApplication([('/tasks/loadworker', MainPage)])__ at the bottom of your __loadworker.py__ file? The error being returned is 404, page not found, which would indicate the request is not being handled. – IanGSY Apr 11 '14 at 16:17
  • This is what I have at the bottom of the page: **application = webapp2.WSGIApplication([('/tasks/loadworker', MainPage)],debug=True)** – user3058197 Apr 11 '14 at 18:49
  • What happens if you go to http://your-app-id.appspot.com/tasks/loadworker in a web browser? – IanGSY Apr 11 '14 at 18:53
  • 1
    Figured it out -- the worker and handler need to be in the same module. I had my worker in app.yaml and my handler in loadandprocess.yaml. Thank you, for the replies, though, I appreciate it. – user3058197 Apr 11 '14 at 18:58
  • 1
    They can be in separate modules. You need to specify the 'target' parameter when you create the task. – Greg Apr 11 '14 at 20:34
  • Thanks Greg -- for anyone searching for this problem later, there are more details here: https://code.google.com/p/googleappengine/issues/detail?id=7738 – user3058197 Apr 11 '14 at 21:13

1 Answers1

2

Loadworker is returning a 404. A taskqueue task must return an http code 200, else it will continue to fail and retry.

Make sure loadworker returns an http response, something like:

return HttpResponse("Did it", mimetype='text/plain')

GAEfan
  • 11,244
  • 2
  • 17
  • 33
  • The end of my page has: **application = webapp2.WSGIApplication([('/tasks/loadworker', MainPage)],debug=True)**. I guess I need to add a return statement somewhere in my def post(self): method? – user3058197 Apr 11 '14 at 18:51
  • I do not think this is the problem here, the request will return 200 status code by default if the page is run successfully, you do not need to set it yourself. – IanGSY Apr 11 '14 at 18:55
  • Figured it out -- the worker and handler need to be in the same module. I had my worker in app.yaml and my handler in loadandprocess.yaml. Thank you, for the replies, though, I appreciate it. – user3058197 Apr 11 '14 at 18:59