Typescript/Angular: Wait for sequentially delayed async tasks to be completed

Question

Even though there's multiple similar questions here on StackOverflow, I haven't been able to solve this, please bear with me:

In an Angular 16 application I have a functionality to batch rename hundreds of users. Thus, I have an array of RenameUser objects, each one having an async method .executeRename() performing an API call to rename a user. To not overwhelm the API, these should be triggered with a 200ms delay between them. Once all users have been renamed, I would like to run other tasks.

Summarized: Trigger a bunch of async operations with a 200ms delay between them, do something when all are complete.

I have managed to delay the calls, and I do not care how long a single call takes to finish, but I need to know when all tasks have finished.

This is the solution I came up with, I would run the triggerRenameProcess() method to start the process.

async triggerRenameProcess(users: RenameUser[]) {
  try {
    await this.renameUsersWithDelay(users, 200);
    console.log('All users renamed');
  } catch (error) {
    console.error('Exception while renaming users: ', error);
  }
}

renameUsersWithDelay(users: RenameUser[], delay: number): Promise<void> {
  return new Promise<void>((resolve) => {
    let currentIndex = 0;
    const executeNext = () => {
      if (currentIndex < users.length) {
        const user = users[currentIndex];
        user.executeRename().then(() => {
          currentIndex++;
          setTimeout(executeNext, delay);
        });
      } else {
        resolve();
      }
    };
  });
}

I kind of understand what the problem is, the renameUsersWithDelay() returns a Promise that can be awaited, but that will happen once all requests have been triggered - it is not waiting for the longest running one to complete. I can make it wait for every single one and run them sequentially, but that's not what it is supposed to do...

I did some research and found this question or this one, but they seem not to address the same issue - and frankly, I am not too familiar with JavaScript - I feel more at home in .NET where I just would use something like .WaitAll() (and probably use a more sophisticated throttling pattern), but I'm not sure how to achieve the same thing in TypeScript.

Answer 1

Whenever you've got an asynchronous data flow manipulation that isn't trivial, you can assume that observables will help you a lot.

Here's the feature you're trying to achieve with observables:

function renameUsersWithDelay(
  users: RenameUser[],
  delayMs: number = 200
): Observable<any> {
  const usersWithDelay = users.map((user, i) =>
    of(user).pipe(delay(delayMs * i), switchMap(executeRename))
  );

  return forkJoin(usersWithDelay);
}

Here's how it works:

• we loop on the users array with a map to create a new array out of it
• for each user, we put the user into an observable using of
• we then apply a delay relative to the index we're at so they're all shifted
• once the delay resolves, we use a switchMap to make make the http call, which will effectively subscribe to the observable returned by executeRename
• because observables are cold by nature, nothing that we just created here is triggered yet. Which is fantastic because it lets you build up all your instructions without triggering them straight away
• finally, we use forkJoin to subscribe to each observables that we just created

I've created a live demo with mocked data so you can see it behaves as expected.

I wanted to give you exactly what you asked for, which is what I did so far. But my advice would be to go for a different approach where you still don't DDOS your backend while being more efficient, by having a limited pool of running calls. For example you could say, up until there's nothing left to proceed, I want to have 3 ongoing rename calls and as soon as 1 finishes in that current pool, another one is taken from the queue and ran straight away. This is IMO more efficient.

Here's how you could implemented this:

const renameUsersWithDelay = (users: RenameUser[]): Observable<any> =>
  from(users.map((user) => of(user).pipe(switchMap(executeRename)))).pipe(
    mergeAll(3)
  );

Live demo