Twisted: How to interrupt callback chain on deferred

Question

My problem: I have a deferred with callbacks and errbacks. I need to stop the process after a specific errback. In another words, if a specific function of errback is called, i need to get its returns and DO NOT process the following callbacks.

from twisted.internet import defer
from twisted.python import failure, util


class Test (object):

@classmethod
def handleFailure(self, f):
    print "handleFailure"
    f.trap(RuntimeError)
    return '0', 'erro'

@classmethod
def handleResult(self, result, message):
    print "handleResult of %s. Old results %s, %s: " % (message, result[0], result[1])
    return 1, 'ok'

@classmethod
def doFailure (self, result, message):
    print "handleResult of %s. Old results %s, %s: " % (message, result[0], result[1])
    raise RuntimeError, "whoops! we encountered an error"


@classmethod
def deferredExample(self):
    d = defer.Deferred()
    # 1o. call without error
    d.addCallback(Test.handleResult, 'call 1')
    # 2o. call without error
    d.addCallback(Test.handleResult, 'call 1')
    # 3o. call causes the failure
    d.addCallback(Test.doFailure,    'call 3')
    # the failure calls the error back 
    d.addErrback (Test.handleFailure) # - A -
    # after error back, the next call back is called
    d.addCallback(Test.handleResult, 'call 4')  # - B -
    # and the last call back is called   
    d.addCallback(Test.handleResult, 'call 5')  # - C -

    d.callback("success")
    return d.result[0], d.result[1]

if __name__ == '__main__': 
#    behindTheScenes("success")
    print "\n-------------------------------------------------\n"
    global num; num = 0
    tst = Test()
    rtn1, rtn2 = tst.deferredExample()
    print "RTN: %s %s" % (rtn1, rtn2)

This code is a simple version than reflects what I need. After -A- process, I need to baypass -B- and -C-, and at the end, the response be "0, erro", not "1, ok'.

My current return:

handleResult of call 1. Old results s, u: 
handleResult of call 1. Old results 1, ok: 
handleResult of call 3. Old results 1, ok: 
handleFailure
handleResult of call 4. Old results 0, erro: 
handleResult of call 5. Old results 1, ok: 
RTN: 1 ok

and I want:

handleResult of call 1. Old results s, u: 
handleResult of call 1. Old results 1, ok: 
handleResult of call 3. Old results 1, ok: 
handleFailure
RTN: 0 erro

How can I do this?

Thanks in advance

that's a really good question, I'd love to hear if anyone has answer for this. There is some interesting discussion of this problem here: http://stackoverflow.com/questions/9295359/stopping-twisted-from-swallowing-exceptions I'm not sure if what you want to do is possible. Basically if you want to stop execution altogether why are you returning result from errback? Perhaps you can just simply raise error in errback? This will stop processing. If you dont raise error Twisted assumes you want to move on and it moves on. — Pawel Miech, May 25 '16 at 16:59
I am using callback chain to control and validate results of http get request (REST web service call) in series. When the result is what I expect, the process continues. When not, I raise failure to call the errback (a method that returns the response modified). But, if in the sequence of calls one fail, the follows are irrelevant. Thanks @PawelMiech for the tip. — Lucas Lima, May 26 '16 at 21:03

score 2 · Accepted Answer · edited May 23 '17 at 12:24

That's a really interesting question. Twisted does not give any documented API for modifying callback chain after error occurs. If you really want to interrupt processing in errback you can always raise error instead of trapping it.

However if you really want to remove all callbacks from Deferred in certain conditions you can use simple hack. Callbacks are kept on Deferred as simple object attributes, see here: https://github.com/twisted/twisted/blob/twisted-16.2.0/twisted/internet/defer.py#L288 if you simply reset this attribute to empty list all callbacks will be removed and processing will be stopped.

@classmethod
def handleFailure(self, f, d):
    print "handleFailure"
    f.trap(RuntimeError)
    d.callbacks = []
    return '0', 'erro'

This works as you'd like it to work, but seems somewhat hacky so I'd be happy to read what other users think. There is more discussion of this problem here: Stopping Twisted from swallowing exceptions discussion there suggests that what you're trying to do is somewhat against the spirit of Twisted.

Better not to do this, you'd better follow the `errback` chain to do this: `return failure in handleFailure to follow the errback chain or return others to follow the callback chain` — Cheney, May 02 '18 at 02:55

Twisted: How to interrupt callback chain on deferred

1 Answers1