4

My epic wakes up when the REMOTE_DATA_STARTED action is dispatched and it fetches data using action.url and action.owner.

I need to make sure I don't initiate two concurrent calls to the same owner/url. Once the call for a owner/url is completed, it's fine to start another one for the same owner/url later.

Cancellation is not what I'm looking for here because I don't want to cancel the existing request(s), I want to prevent starting a new requests.

I feel like I need a mix of exhaustMap and groupBy, but I don't know where to go from here.

This is my epic at this point, it rejects all concurrent calls, not by owner/url

const myEpic = action$ =>
  action$.ofType("REMOTE_DATA_STARTED").exhaustMap(action =>
    fakeAjaxCall().map(() => {
      return { type: "COMPLETED", owner: action.owner, url: action.url };
    })
  );

Try it Live

I created this test projet with a failing test case. Can you help me make this work?

https://codesandbox.io/s/l71zq6x8zl

As you will see, test1_exhaustMapByActionType_easy works fine, it's test2_exhaustMapByActionTypeOwnerAndUrl that fails.

Make sure you expand the console to see the test results.

Whymarrh
  • 13,139
  • 14
  • 57
  • 108
Sylvain
  • 19,099
  • 23
  • 96
  • 145
  • 2
    If I'm understanding the problem correctly, you're going to need some state to keep track of what pending requests you have by the owner/url pair. How particular are you about expressing this in terms of the existing operators? It might be easiest/cleanest to keep a separate mapping of owners/urls to subjects and then feed actions into that (e.g. one subject per key, call next with the actions, exhaustMap those, merge all of those back together). – Whymarrh Mar 29 '18 at 19:03
  • 1
    Here is the exact code for your problem: https://codesandbox.io/s/vyloq48xoy [miles_christian](https://stackoverflow.com/users/9468492/miles-christian) corrected the test cases also to remove timing issues. – wp78de Mar 30 '18 at 23:27
  • @wp78de if you add your own answer with your solution, I'll accept your answer. You code is not the same as ZahiC's and although his answer might also work, your code is easier to understand (at least for me). – Sylvain Apr 02 '18 at 17:55
  • @Sylvain my pleasure! – wp78de Apr 02 '18 at 19:32

3 Answers3

2

It's surely can be done with groupBy & exhastMap in an elegant way:

const groupedByExhaustMap = (keySelector, project) => 
  source$ => source$.pipe(
    groupBy(keySelector),
    mergeMap(groupedCalls => 
      groupedCalls.pipe(
        exhaustMap(project)
      )
    )
  );

const { delay, groupBy, mergeMap, exhaustMap } = Rx.operators;

const groupedByExhaustMap = (keySelector, project) => 
  source$ => source$.pipe(
    groupBy(keySelector),
    mergeMap(groupedCalls => 
      groupedCalls.pipe(
        exhaustMap(project)
      )
    )
  );

const calls = [ // every call takes 500ms
  {startTime: 0, owner: 1, url: 'url1'}, 
  {startTime: 200, owner: 2, url: 'url2'},
  {startTime: 400, owner: 1, url: 'url1'}, // dropped
  {startTime: 400, owner: 1, url: 'url2'},
  {startTime: 600, owner: 1, url: 'url1'}
];

const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));

const simulateCallsOverTime$ = Rx.Observable.from(calls)  
  .pipe(
    mergeMap(call => Rx.Observable.of(call)
      .pipe(
        delay(call.startTime)
      )
    )
  );

simulateCallsOverTime$
  .pipe(
    groupedByExhaustMap(
      call => `${call.owner}_${call.url}`,
      async call => {
        await sleep(500); // http call goes here
        return call;
      }
    )
  )
  .subscribe(console.log);
<script src="https://cdnjs.cloudflare.com/ajax/libs/rxjs/5.5.8/Rx.js"></script>
ZahiC
  • 13,567
  • 3
  • 25
  • 27
2

There are many ways to approach a solution. My first thought was that you could split the actions into owner+URL subjects and then work with those:

const myEpic = (action$) => {
    const completed$ = new Subject();
    const flights = new DefaultMap((pair$) =>
        pair$.exhaustMap((action) =>
            fakeAjaxCall().map(() => ({
                ...action,
                type: 'COMPLETED',
            }))
        )
        .subscribe((action) => completed$.next(action))
    );
    action$.ofType('REMOTE_DATA_STARTED')
        .subscribe((action) => {
            flights.get(`${action.owner}+${action.url}`).next(action);
        });

    return completed$;
};

That works, but admittedly it requires maintaining some sort of "default map" where new owner+URL pairs get a new Subject (I wrote a quick implementation). A test case that it passes:

test('myEpic does both drop actions and NOT drop actions for two owner+url pairs', async () => {
    const arrayOfAtMost = (action$, limit) => action$.take(limit)
        .timeoutWith(1000, Observable.empty())
        .toArray().toPromise();
    const action$ = new ActionsObservable(
        Observable.create((observer) => {
            // Jim #1 emits four (4) concurrent calls—we expect only two to be COMPLETED, one per URL
            observer.next({ type: 'REMOTE_DATA_STARTED', url: 'google.com', owner: 'jim1' });
            observer.next({ type: 'REMOTE_DATA_STARTED', url: 'google.com', owner: 'jim1' });
            observer.next({ type: 'REMOTE_DATA_STARTED', url: 'google.org', owner: 'jim1' });
            observer.next({ type: 'REMOTE_DATA_STARTED', url: 'google.org', owner: 'jim1' });

            // Jim #2 emits two (2) calls at the same time as Jim #1—we expect only one to be COMPLETED, deduped URLs
            observer.next({ type: 'REMOTE_DATA_STARTED', url: 'google.biz', owner: 'jim2' });
            observer.next({ type: 'REMOTE_DATA_STARTED', url: 'google.biz', owner: 'jim2' });

            // Once all of the above calls are completed, Jim #1 and Jim #2 make calls simultaneously
            // We expect both to be COMPLETED
            setTimeout(() => {
                const url = 'https://stackoverflow.com/q/49563059/1267663';
                observer.next({ type: 'REMOTE_DATA_STARTED', url, owner: 'jim1' });
                observer.next({ type: 'REMOTE_DATA_STARTED', url, owner: 'jim2' });
            }, 505);
        })
    );
    const resultant$ = myEpic(action$);
    const results = await arrayOfAtMost(resultant$, 5);

    expect(results).toEqual([
        { type: 'COMPLETED', url: 'google.com', owner: 'jim1' },
        { type: 'COMPLETED', url: 'google.org', owner: 'jim1' },
        { type: 'COMPLETED', url: 'google.biz', owner: 'jim2' },
        { type: 'COMPLETED', url: 'https://stackoverflow.com/q/49563059/1267663', owner: 'jim1' },
        { type: 'COMPLETED', url: 'https://stackoverflow.com/q/49563059/1267663', owner: 'jim2' },
    ]);
});

The full solution, including the DefaultMap implementation:

const { Observable, Subject } = require('rxjs');

class DefaultMap extends Map {
    constructor(initializeValue) {
        super();
        this._initializeValue = initializeValue || (() => {});
    }

    get(key) {
        if (this.has(key)) {
            return super.get(key);
        }

        const subject = new Subject();
        this._initializeValue(subject);
        this.set(key, subject);
        return subject;
    }
}

const fakeAjaxCall = () => Observable.timer(500);
const myEpic = (action$) => {
    const completed$ = new Subject();
    const flights = new DefaultMap((uniquePair) =>
        uniquePair.exhaustMap((action) =>
            fakeAjaxCall().map(() => ({
                ...action,
                type: 'COMPLETED',
            }))
        )
        .subscribe((action) => completed$.next(action))
    );
    action$.ofType('REMOTE_DATA_STARTED')
        .subscribe((action) => {
            flights.get(`${action.owner}+${action.url}`).next(action);
        });

    return completed$;
};

* The above snippet isn't actually runnable, I just wanted it collapsible.

Runnable example with test cases

I've put together a runnable example with test cases on GitHub.

Using groupBy and exhaustMap operators

I wrote up the original solution with tests only to discover that, yes, it is possible with existing operators, the groupBy and exhaustMap you suggested:

const myEpic = action$ =>
    action$.ofType('REMOTE_DATA_STARTED')
        .groupBy((action) => `${action.owner}+${action.url}`)
        .flatMap((pair$) =>
            pair$.exhaustMap(action =>
                fakeAjaxCall().map(() => ({
                    ...action,
                    type: 'COMPLETED',
                }))
            )
        );

Running that against the same test suite above will pass.

Whymarrh
  • 13,139
  • 14
  • 57
  • 108
1

Okay, here we go:

GroupBy req.owner, flatten the results:

const myEpic = action$ =>
  action$
    .ofType("REMOTE_DATA_STARTED")
    .groupBy(req => req.owner)
    .flatMap(ownerGroup => ownerGroup.groupBy(ownerReq => ownerReq.url))
    .flatMap(urlGroup => 
      urlGroup.exhaustMap(action => 
        fakeAjaxCall().map(() => ({ type: "COMPLETED", owner: action.owner, url: action.url }))
      )
    )

Don't forget the observe.complete();

const test1_exhaustMapByActionType_easy = () => {
  const action$ = new ActionsObservable(
    Observable.create(observer => {
      observer.next({ type: "REMOTE_DATA_STARTED", owner: "ownerX", url: "url1" });
      observer.next({ type: "REMOTE_DATA_STARTED", owner: "ownerX", url: "url1" });
      setTimeout(() => {
        observer.next({ type: "REMOTE_DATA_STARTED", owner: "ownerX", url: "url1" });
        observer.complete();
      }, 30);
    })
  );

  const emittedActions = [];
  const epic$ = myEpic(action$);

  epic$.subscribe(action => emittedActions.push(action), null, () => expect("test1_exhaustMapByActionType_easy", 2, emittedActions));
};

Same here:

const test2_exhaustMapByActionTypeOwnerAndUrl = () => {
  const action$ = new ActionsObservable(
    Observable.create(observer => {
      // owner1 emmits 4 concurrent calls, we expect only two to COMPLETED actions; one per URL:
      observer.next({ type: "REMOTE_DATA_STARTED", owner: "owner1", url: "url1" });
      observer.next({ type: "REMOTE_DATA_STARTED", owner: "owner1", url: "url1" });
      observer.next({ type: "REMOTE_DATA_STARTED", owner: "owner1", url: "url2" });
      observer.next({ type: "REMOTE_DATA_STARTED", owner: "owner1", url: "url2" });

      // owner2 emmits 2 calls at the same time as owner 1. because the two calls
      // from owner2 have the same url, we expecty only one COMPLETED action
      observer.next({ type: "REMOTE_DATA_STARTED", owner: "owner2", url: "url1" });
      observer.next({ type: "REMOTE_DATA_STARTED", owner: "owner2", url: "url1" });

      // Once all of the above calls are completed each owner makes one concurrent call
      // we expect each call to go throught and generate a COMPLETED action
      setTimeout(() => {
        observer.next({ type: "REMOTE_DATA_STARTED", owner: "owner1", url: "url1" });
        observer.next({ type: "REMOTE_DATA_STARTED", owner: "owner2", url: "url1" });
        observer.complete();
      }, 30);
    })
  );

  const emittedActions = [];
  const epic$ = myEpic(action$);

  epic$.subscribe(action => emittedActions.push(action), null, () => expect("test2_exhaustMapByActionTypeOwnerAndUrl", 5, emittedActions));
};

Full sample

wp78de
  • 18,207
  • 7
  • 43
  • 71
  • 1
    Note that completing the action stream `action$` (calling `observer.complete();`) isn't correct because redux-observable doesn't complete the stream—`action$` is an infinite hot observable – Whymarrh Apr 03 '18 at 14:01
  • 1
    Also note that `flatMap`-ing the stream twice in the epic is unnecessary – Whymarrh Apr 03 '18 at 14:03