Generic approach to deferred all?

Question

I need something in twistd similar to JS' Promise.all(). I found this FireWhenAllFinish example, but it is buggy (throws exceptions, maybe outdated).

This is not really a question, cause I found a solution, see below. Still posting it here so it might help others.

score 0 · Answer 1 · answered Jan 12 '18 at 00:05

I came up with this in the end:

class DeferredAll(Deferred):
    def __init__(self, deferreds):
        super(DeferredAll, self).__init__()
        assert deferreds, 'Need at least one deferred for DeferredAll'
        self.deferreds = deferreds
        self._results = []
        self.finished_count = 0

        for d in self.deferreds:
            d.addCallbacks(self._success, self._fail)

    def _success(self, result):
        self.finished_count += 1
        # print('_success {}/{} {}'.format(self.finished_count, len(self.deferreds), result))
        self._results.append(result)
        if self.finished_count == len(self.deferreds):
            self.callback(self._results)

    def _fail(self, *args, **kwargs):
        # print('_fail {} {}'.format(str(args), str(kwargs)))
        if not self.called: # this property is True if callback()/errback() has already been called
            self.failed = True
            self.errback()

score 0 · Answer 2 · answered Jan 12 '18 at 12:26

There are a couple solutions for this in Twisted:

from __future__ import print_function

from twisted.internet.defer import DeferredList, gatherResults, succeed

DeferredList([succeed(1), succeed(2)]).addCallback(print)
gatherResults([succeed(1), succeed(2)]).addCallback(print)

Generic approach to deferred all?

2 Answers2