rxjs chain can be triggered parallel but internal logic should be synchrounous (.next with concat)

Question

Fields in my case can be finalized (actions linked to a field were executed). When they are done, I need to update 2 lists:

alreadyExecutedFields: string[] --> plain array
remainingFieldsToExecute: BehaviorSubject<string[]> --> behavior subject, because a .next needs to trigger other logic. This logic can be triggered parallel, but I want to prevent that because there's a splice within this logic which can behave incorrect then. (splicing an index, that was removed in the parallel chain.)

So when a field needs to be finalized, I call:

this.finalize$.next(field);

And the finalize$ chain looks like this:

this.finalize$.pipe(
        concatMap((field: string) => {
            return new Promise<void>((resolve) => {
                console.log('START', field);
                this.alreadyExecutedFields.push(field);
                const remainingFieldsToExecute = this.remainingFieldsToExecute$.value;
                remainingFieldsToExecute.splice(remainingFieldsToExecute.indexOf(field), 1);
                this.remainingFieldsToExecute$.next(remainingFieldsToExecute);
                console.log('END', field);
                resolve();
            });
        }),
    ).subscribe(() => { });

But for some reason, when 2 finalize$.next calls happen right after each other, the concatMap doesn't await the promise of the previous one. Also when I tried to put a timeout around the END log and the resolve, it doesn't await the previous resolve.

What does work in my case is instead of using a concatMap, using a setInterval with a flag, which locks the part of the code where the lists are being updated.

But how can this be done in a better way? Or in a correct way with or without concat pipes.

It's not clear to me how do you know that `concatMap` is not waitng for the promise to resolve. What issue are you facing exaclty? — lbsn, Sep 13 '21 at 14:36
If I add logs in the promise function: console.log('START', field); // on the first line console.log('END', field); // right before resolve I see the start log of some fields already before the end of a previous field is logged. — jmeire, Sep 13 '21 at 14:56
I can't reproduce that behaviour: [stackblitz](https://stackblitz.com/edit/angular-ivy-1ebp5m). Maybe your actual code implements some logic that you're not sharing in your question? — lbsn, Sep 13 '21 at 15:30
The way that the finalize$.next can be triggered is by listening to the remainingFieldsToExecute$ behaviorSubject. So it could possible be that 3 fields simultaneously call finalize$.next. Is that a problem for the splice logic part? Because that's my fear, that these will interfere and that a splice happens with incorrect indexes because the splice logic is being executed in parallel. So I wanted to validate that by setting a timeOut around the resolve, to validate that the next finalize awaits the timeout of the previous one. This would confirm for me that the system is waterproof. — jmeire, Sep 14 '21 at 07:12
Updated my stackblitz and still dont' see any problem. But it's hard to grasp a clear idea without seing more of your code, I bet your actual use case is more complex. A working example would probably help. Anyway, I doubt that your splice logic is really executed in parallel: `concatMap` will definitely wait for the Promise to resolve before merging the next value. But note that your Promise resolve immediately after `remainingFieldsToExecute$.next()`, so it won't wait for any logic triggered by that call. — lbsn, Sep 14 '21 at 07:50
I tried it on your stackblitz with some timeOuts and I could not reproduce it... after some investigation I discovered a difference in rxjs version. In my application I still use 6.5.5 (+ Angular v11). When I changed the stackblitz example to rxjs 6.5.5 it also went wrong! So I guess I will need to plan to update my dependencies to resolve the problem and make concatMap behave as expected. Thank you for the investigation! — jmeire, Sep 14 '21 at 13:04
This is my updated [stackblitz](https://stackblitz.com/edit/angular-ivy-jdu19s?file=src%2Fapp%2Fapp.component.ts). I have another optimization question: Now in the finalizeFieldExecution method, after the finalize$.next, I still need to resolve the promise. Now I do this by subscribing again on the list to check if it's filtered out. Can I do this in a better way? — jmeire, Sep 14 '21 at 13:14

shutsman · Answer 1 · 2021-09-13T16:02:57.803

to modify alreadyExecutedFields you can use tap operator that used for side-effects
to extract value from remainingFieldsToExecute you can use withLatestFrom
then to modify remainingFieldsToExecute - you can again use tap

  alreadyExecutedFields: string[] = [];
  remainingFieldsToExecute$: BehaviorSubject<string[]> = new BehaviorSubject<
    string[]
  >(['1', '2', '3', '4', '5']);
  finalize$ = new BehaviorSubject('1');

  ngOnInit() {
    this.finalize$
      .pipe(
        tap(field => this.alreadyExecutedFields.push(field)),
        withLatestFrom(this.remainingFieldsToExecute$),
        tap(([field, remainings]) => {
          remainings.splice(remainings.indexOf(field), 1);
          this.remainingFieldsToExecute$.next(remainings);
        })
      )
      .subscribe(([_, data]) => {
        console.group();
        console.log('*** NEW EMIT ***');
        console.log('current field:', _);
        console.log('remainingFieldsToExecute', data);
        console.log('alreadyExecutedFields:', this.alreadyExecutedFields);
        console.groupEnd();
      });

    this.finalize$.next('2');
    this.finalize$.next('3');
  }

demo: https://stackblitz.com/edit/angular-ivy-ensh7w?file=src/app/app.component.ts

logs:

It could be smplified to:


    this.finalize$
      .pipe(
        mergeMap(field =>
          this.remainingFieldsToExecute$.pipe(
            map(remFields => {
              this.alreadyExecutedFields.push(field);
              remFields.splice(remFields.indexOf(field), 1);
              return remFields;
            })
          )
        )
      )
      .subscribe()

In my code finalize$.next will trigger by listening to the remaining..$ subject. So could be that 3 fields simultaneously call finalize$.next. Is that a problem for splice logic part? Because that's my fear, that these will interfere and a splice happens with an incorrect index cause the splice logic is being executed in parallel. Is it possible that a field will be removed from the array, in a parallel chain another field will be removed, so both intermediate results will be different, because a different field is removed from the array. It has to be nexted again on the remaining..$ subject. — jmeire, Sep 14 '21 at 07:21
honestly, in JS all code cannot happen simultaneously, in any case, we have a queue even for async operation) in this case use `find` and `filter`, if field is found then filter data and call `next`, if not found you should not call next — shutsman, Sep 14 '21 at 07:34

rxjs chain can be triggered parallel but internal logic should be synchrounous (.next with concat)

1 Answers1