4

I'm new to Twisted and after finally figuring out how the deferreds work I'm struggling with the tasks. What I want to achieve is to have a script that sends a REST request in a loop, however if at some point it fails I want to stop the loop. Since I'm using callbacks I can't easily catch exceptions and because I don't know how to stop the looping from an errback I'm stuck.

This is the simplified version of my code:

def send_request():
    agent = Agent(reactor)
    req_result = agent.request('GET', some_rest_link)
    req_result.addCallbacks(cp_process_request, cb_process_error)

if __name__ == "__main__":
    list_call = task.LoopingCall(send_request)
    list_call.start(2)
    reactor.run()
Michal
  • 6,411
  • 6
  • 32
  • 45

1 Answers1

4

To end a task.LoopingCall all you need to do is call the stop on the return object (list_call in your case).

Somehow you need to make that var available to your errback (cb_process_error) either by pushing it into a class that cb_process_error is in, via some other class used as a pseudo-global or by literally using a global, then you simply call list_call.stop() inside the errback.

BTW you said:

Since I'm using callbacks I can't easily catch exceptions

Thats not really true. The point of an errback to to deal with exceptions, thats one of the things that literally causes it to be called! Check out my previous deferred answer and see if it makes errbacks any clearer.

The following is a runnable example (... I'm not saying this is the best way to do it, just that it is a way...)

#!/usr/bin/python

from twisted.internet import task
from twisted.internet import reactor
from twisted.internet.defer import Deferred
from twisted.web.client import Agent
from pprint import pprint

class LoopingStuff (object):

    def cp_process_request(self, return_obj):
        print "In callback"
        pprint (return_obj)

    def cb_process_error(self, return_obj):
        print "In Errorback"
        pprint(return_obj)
        self.loopstopper()

    def send_request(self):
        agent = Agent(reactor)
        req_result = agent.request('GET', 'http://google.com')
        req_result.addCallbacks(self.cp_process_request, self.cb_process_error)

def main():
    looping_stuff_holder = LoopingStuff()
    list_call = task.LoopingCall(looping_stuff_holder.send_request)
    looping_stuff_holder.loopstopper = list_call.stop
    list_call.start(2)
    reactor.callLater(10, reactor.stop)
    reactor.run()

if __name__ == '__main__':
  main()

Assuming you can get to google.com this will fetch pages for 10 seconds, if you change the second arg of the agent.request to something like http://127.0.0.1:12999 (assuming that port 12999 will give a connection refused) then you'll see 1 errback printout (which will have also shutdown the loopingcall) and have a 10 second wait until the reactor shuts down.

Community
  • 1
  • 1
Mike Lutz
  • 1,812
  • 1
  • 10
  • 17
  • I know about the stop method, however the errback is also called when the connection has been terminated. Is there an easy way to differentiate between a successful connection with its termination and a failed connection in an errback? – Michal May 24 '14 at 17:10
  • the errback is called on close??? You need to provide some more code in your question. Normally a close just calls your `protocol`s `connectionLost` and doesn't fire a deferred, I'm confused how that is happening in your case. You can see the style of close I'm talking about in the example code in [this](http://stackoverflow.com/a/23274411/3334178) prev answer – Mike Lutz May 24 '14 at 17:18
  • Wait, are you exclusively talking about the `agent.request` call? (I.E. your not doing connections yourself with `listenTCP` or the like) I was giving a more genaric answer for code in twisted. – Mike Lutz May 24 '14 at 17:26
  • Yes, I'm talking exclusively about agent.request. Please ignore my comment about the errback firing when the connection has been closed, it was my bug. – Michal May 24 '14 at 17:37
  • Yup, just wrote sample code and I'm only getting errback on connection error, otherwise *only* getting callback. I'll add code to my answer in a minute. – Mike Lutz May 24 '14 at 17:43
  • I've finally had some time to implement your answer and it works great. I really like how you utilized python features. – Michal May 27 '14 at 12:24