How to reset a RXJS scan operator based on another Observable

Question

I have a component which triggers an onScrollEnd event when the last item in a virtual list is rendered. This event will do a new API request to fetch the next page and merge them with the previous results using the scan operator.

This component also has a search field which triggers an onSearch event.

How do I clear the previous accumulated results from the scan operator when a search event is triggered? Or do I need to refactor my logic here?

const loading$ = new BehaviorSubject(false);
const offset$ = new BehaviorSubject(0);
const search$ = new BehaviorSubject(null);

const options$: Observable<any[]> = merge(offset$, search$).pipe(
  // 1. Start the loading indicator.
  tap(() => loading$.next(true)),
  // 2. Fetch new items based on the offset.
  switchMap(([offset, searchterm]) => userService.getUsers(offset, searchterm)),
  // 3. Stop the loading indicator.
  tap(() => loading$.next(false)),
  // 4. Complete the Observable when there is no 'next' link.
  takeWhile((response) => response.links.next),
  // 5. Map the response.
  map(({ data }) =>
    data.map((user) => ({
      label: user.name,
      value: user.id
    }))
  ),
  // 6. Accumulate the new options with the previous options.
  scan((acc, curr) => {
    // TODO: Dont merge on search$.next 
    return [...acc, ...curr]);
  }
);

// Fetch next page
onScrollEnd: (offset: number) => offset$.next(offset);
// Fetch search results
onSearch: (term) => {
  search$.next(term);
};

Answer 1

To manipulate the state of a scan you can write higher order functions that get the old state and the new update. Combine then with the merge operator. This way you stick to a clean stream-oriented solution without any side-effects.

const { Subject, merge } = rxjs;
const { scan, map } = rxjs.operators;

add$ = new Subject();
clear$ = new Subject();

add = (value) => (state) => [...state, value];
clear = () => (state) => [];

const result$ = merge(
  add$.pipe(map(add)),
  clear$.pipe(map(clear))
).pipe(
  scan((state, innerFn) => innerFn(state), [])
)

result$.subscribe(result => console.log(...result))

add$.next(1)
add$.next(2)
clear$.next()
add$.next(3)

<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/6.5.3/rxjs.umd.min.js"></script>

This method can easily be extended and/or adapted for other state usecases in rxjs.

Example (remove last item)

removeLast$ = new Subject()

removeLast = () => (state) => state.slice(0, -1);

merge(
  ..
  removeLast$.pipe(map(removeLast)),
  ..
)

Answer 2

I think you could achieve what you want just by restructuring your chain (I'm omitting tap calls that trigger loading for simplicity):

search$.pipe(
  switchMap(searchterm =>
    concat(
      userService.getUsers(0, searchterm),
      offset$.pipe(concatMap(offset => userService.getUsers(offset, searchterm)))),
    ).pipe(
      map(({ data }) => data.map((user) => ({
        label: user.name,
        value: user.id
      }))),
      scan((acc, curr) => [...acc, ...curr], []),
    ),
  ),
);

Every emission from search$ will create a new inner Observable with its own scan that will start with an empty accumulator.

Answer 3

Found a working solution: I check the current offset by using withLatestFrom before the scan operator and reset the accumulator if needed based on this value.

Stackblitz demo

Answer 4

This is an interesting stream. Thinking about it, offset$ and search$ are really 2 separate streams, though, with different logic, and so should be merged at the very end and not the beginning.

Also, it seems to me that searching should reset the offset to 0, and I don't see that in the current logic.

So here's my idea:

const offsettedOptions$ = offset$.pipe(
    tap(() => loading$.next(true)),    
    withLatestFrom(search$),
    concatMap(([offset, searchterm]) => userService.getUsers(offset, searchterm)),
    tap(() => loading$.next(false)),
    map(({ data }) =>
    data.map((user) => ({
      label: user.name,
      value: user.id
    })),
    scan((acc, curr) => [...acc, ...curr])
);

const searchedOptions$ = search$.pipe(
    tap(() => loading$.next(true)),
    concatMap(searchTerm => userService.getUsers(0, searchterm)),
    tap(() => loading$.next(false)),
    map(({ data }) =>
    data.map((user) => ({
      label: user.name,
      value: user.id
    })),
);

const options$ = merge(offsettedOptions, searchedOptions);

See if that works or would make sense. I may be missing some context.

Answer 5

I know its old, but I just needed to do the same thing and have another solution to throw in to the mix.

There are really just 2 actions the user can trigger

const search$ = new Subject<string>();
const offset$ = new Subject<number>();

We don't really care about offset$ until search$ emits, and at that point, we want it to be 0 to start over. I would write it like this:

const results$ = search$.pipe( // Search emits
  switchMap((searchTerm) => {
    return offset$.pipe( // Start watching offset
      startWith(0),  // We want a value right away, so set it to 0
      switchMap((offset) => {
        return userService.getUsers(offset, searchTerm)) // get the stuff
      })
    )
  }))

At this point we are resetting the offset every time search$ emits, and any time offset$ emits we make a fresh api call fetching the desired resources. We need the collection to reset if search$ emits, so I believe the right place is inside switchMap wrapping the offset$ pipe.

const results$ = search$.pipe( // Search emits
  switchMap((searchTerm) => {
    return offset$.pipe( // Start watching offset
      startWith(0),  // We want a value right away, so set it to 0
      switchMap((offset) => {
        return userService.getUsers(offset, searchTerm)) // get the stuff
      }),
      takeWhile((response) => response.links.next), // stop when we know there are no more.
      // Turn the data in to a useful shape
      map(({ data }) => 
        data.map((user) => ({
          label: user.name,
          value: user.id
        }))
      ),
      // Append the new data with the existing list
      scan((list, response) => {
        return [  // merge
          ...list,
          ...response
        ]
      }, [])
    )
  }))

This great part here is that the scan is reset on every new search$ emission.

The final bit here, I would move loading$ out of tap, and declare it separately. Final code should look something like this

const search$ = new Subject<string>();
const offset$ = new Subject<number>();
let results$: Observable<{label: string; value: string}[]>;

results$ = search$.pipe( // Search emits
  switchMap((searchTerm) => {
    return offset$.pipe( // Start watching offset
      startWith(0),  // We want a value right away, so set it to 0
      switchMap((offset) => {
        return userService.getUsers(offset, searchTerm)) // get the stuff
      }),
      takeWhile((response) => response.links.next), // stop when we know there are no more.
      // Turn the data in to a useful shape
      map(({ data }) => 
        data.map((user) => ({
          label: user.name,
          value: user.id
        }))
      ),
      // Append the new data with the existing list
      scan((list, response) => {
        return [  // merge
          ...list,
          ...response
        ]
      }, [])
    )
  }));

const loading$ = merge(
  search$.pipe(mapTo(true)), // set to true whenever search emits
  offset$.pipe(mapTo(true)),  // set to true when offset emits
  results$.pipe(mapTo(false)), // set to false when we get new results
);

results$.subscribe((results) => {
  console.log(results);
})