RxJS: Cache Several XHR Calls without Mutability and Side Effects

Question

I’m looking for an RxJS example how to cache a series of XHR calls (or other async operations), so the same call does not have to be repeated, while respecting immutability and with no side effects.

Here's a bare-bones, mutable example:

var dictionary = {}; // mutable

var click$ = Rx.Observable.fromEvent(document.querySelector('button'), 'click', function (evt) {
  return Math.floor(Math.random() * 6) + 1; // click -> random number 1-6 (key)
})
  .flatMap(getDefinition);

var clicksub = click$.subscribe(function (key) {
  console.log(key);
});

function getDefinition (key) {
  if ( dictionary[key] ) { // check dict. for key
    console.log('from dictionary');
    return Rx.Observable.return(dictionary[key]);
  }
  // key not found, mock up async operation, to be replaced with XHR
  return Rx.Observable.fromCallback(function (key, cb) {
    dictionary[key] = key; // side effect
    cb(dictionary[key); // return definition
  })(key);
}

JSBin Demo

Question: Is there a way to accomplish caching several similar async operations without resorting to using the dictionary variable, due to mutability and side effect?

---

I’ve looked at scan as a means to “collect” the XHR call results, but I don’t see how to handle an async operation within scan.

I think I’m dealing with two issues here: one is state management maintained by the event stream rather than kept in a variable, and the second is incorporating a conditional operation that may depend on an async operation, in the event stream flow.

Answer 1

Using the same technique than in a previous question (RxJS wait until promise resolved), you could use scan and add the http call as a part of the state you keep track of. This is untested but you should be able to easily adapt it for tests :

restCalls$ = click$
  .scan(function (state, request){
    var cache = state.cache;
    if (cache.has(request)) {
      return {cache : cache, restCallOrCachedValue$ : Rx.Observable.return(cache.get(request))}
    }
    else {
      return {
        cache : cache,
        restCallOrCachedValue$ : Rx.Observable
          .defer(function(){
            return Rx.Observable
              .fromPromise(executeRestCall(request))
              .do(function(value){cache.add(request,value)})
          })
      }
    }
  }, {cache : new Cache(), restCallOrCachedValue$ : undefined})
  .pluck('restCallOrCachedValue$')
  .concatAll()

So basically you pass an observable which does the call down the stream or you directly return the value in the cache wrapped in an observable. In both cases, the relevant observables will be unwrapped in order by concatAll. Note how a cold observable is used to start the http call only at the time of the subscription (supposing that executeRestCall executes the call and returns immediately a promise).

Answer 2

Working from @user3743222's response, this code does cache the XHR calls:

// set up Dictionary object
function Dictionary () { this.dictionary = {} }
Dictionary.prototype.has = function (key) { return this.dictionary[key] }
Dictionary.prototype.add = function (key, value) { this.dictionary[key] = value }
Dictionary.prototype.get = function (key) { return this.dictionary[key] }

var definitions$ = click$

  .scan(function (state, key) {
    var dictionary = state.dictionary

    // check for key in dict.
    if ( dictionary.has(key) ) {
      console.log('from dictionary')
      return {
        dictionary: dictionary, 
        def$: Rx.Observable.return(dictionary.get(key))
      }
    }

    // key not found
    else {
      return {
        dictionary: dictionary, 
        def$: Rx.Observable.fromPromise(XHRPromise(key))
          .do(function (definition) { dictionary.add(key, definition) })
      }
    }

  }, {dictionary : new Dictionary(), def$ : undefined})

  .pluck('def$') // pull out definition stream
  .concatAll()   // flatten

Updated: In the // key not found block, executing the function XHRPromise (that returns a promise) passing in the key and the result passed to the RxJS method fromPromise. Next, chain a do method in which the definition is grabbed and added to the dictionary cache.

Follow up question: It appears we've removed the side effect issue, but is this still considered mutating when the definition and key are added to the dictionary?

---

Putting this here for archival purposes, the first iteration had this code, accomplishing the same caching procedure:

    // ALTERNATIVE key not found
    else {
      var retdef = Rx.Observable.fromPromise(function () {
        var prom = XHRPromise(key).then(function (dd) {
          dictionary.add(key, dd) // add key/def to dict.
          return dd
        })
        return prom
      }()) // immediately invoked anon. func.
      return { dictionary: dictionary, def$: retdef }
    }

Here, using the function XHRPromise that returns a promise. Before returning that promise to the event stream, though, pull out a then method in which the promised definition is grabbed and added to the dictionary cache. After that the promise object is returned to the event stream through the RxJS method fromPromise.

To get access to the definition returned from the XHR call, so it can be added to the dictionary cache, an anonymous, immediately invoked function is used.