Custom for..of using queue to solve concurrency in promises yields slower requests than Promise.all

Question

here is my problem: I have a stack of x promises that I want to solve 5 at a time (basically what Plimit does)

Here is the code that I implemented using for...of

async queuefy(items, concurrency, callback) {
    console.log(items.length);
    let queue = [];
    let singleQueue = [];
    let now = 0;
    items.forEach((item, index) => {
        now++;
        singleQueue.push(item);
        if(now === concurrency || index === items.length - 1) {
            queue.push(singleQueue);
            now = 1;
            singleQueue = [];
        }
    });
    let batch = 0;
    let ret = [];
    for(const que of queue) {
        const currentRes = await Promise.all(que.map(async (q) => {
            return await callback(q);
        }));
        console.log("Resolved batch: ", ++batch);
        ret.push(...currentRes);
    }
    ret = ret.filter(ret => ret !== undefined);
    return ret;
}

Also having havier logic to solve, the time increases from using good ol' Promise.all. Only tested for up to 15 instances of "queuefy".

Is something wrong with my code or will the request time number lower from using Promise.all for larger examples ?

Did you want to use always 5 at a time or do you want a limit of 5 thread at a time ? I already wrote something that executes multiple threads asynchronicaly with a limit of x threads... i can search it if you want. — Patrick Ferreira, Feb 28 '22 at 17:11
"*the time increases from using good ol' Promise.all.*" - and what is unexpected about that? Obviously queuing batches of 5 will take longer than doing all 15 concurrently? — Bergi, Feb 28 '22 at 17:16
Does this answer your question? [Throttle amount of promises open at a given time](https://stackoverflow.com/questions/38385419/throttle-amount-of-promises-open-at-a-given-time) — Samathingamajig, Feb 28 '22 at 19:45

score 1 · Accepted Answer · answered Feb 28 '22 at 19:43

It should not be too surprising that, if choosing between requesting 15 items simultaneously and requesting at most 5 requests at a time, the throttled/queued implementation takes longer. What had been done in parallel is now being done semi-serially. Presumably the use of throttling or serial behavior here is not to absolutely save wall-clock time, but rather to even out server load or allow the browser/network to prioritize other connections within its own limit.

One aspect of your code that you might not have expected, however: Your Promise.all batches of 5 will now take as long as the longest request before the next batch starts. Consequently, just before a new batch starts, it is likely that far fewer than 5 requests are currently open.

(A1-------)             (B1-----------)(C1--)                 |
(A2---------)           (B2--)         (C2-------------------)|
(A3---)                 (B3-------)    (C3------)             | done
(A4--------------------)(B4----)       (C4----------)         |
(A5-----)               (B5---------)  (C5----)               |

TIME->--------------------------------------------------------|

In other implementations like PLimit and mine on SO, there are no intermediate calls to Promise.all, so you can complete those calls more optimally while also ensuring that no more than 5 open promises exist at once.

( 1-------)( 8-----------)(14--)            |
( 2---------)( 9--)(11-------------------)  |
( 3---)( 6-------)(10------)                | done
( 4--------------------)(13----)            |
( 5-----)( 7---------)(12----)(15----------)|

TIME->--------------------------------------|

For this reason, you may want to abandon the use of Promise.all to better keep your open channels/threads saturated.

Custom for..of using queue to solve concurrency in promises yields slower requests than Promise.all

1 Answers1