1

I am trying to improve performance of querying a couchbase view by using async gets. I have read their documentation about the proper way to do so, it goes something like:

Cluster cluster = CouchbaseCluster.create();
Bucket bucket = cluster.openBucket();


List<JsonDocument> foundDocs = Observable
.just("key1", "key2", "key3", "key4", "key5")
.flatMap(new Func1<String, Observable<JsonDocument>>() {
    @Override
    public Observable<JsonDocument> call(String id) {
        return bucket.async().get(id);
    }
})
.toList()
.toBlocking()
.single();

Which works great and fast, but since I rely on the order of the results, it seems that i need to do some extra work to keep the results ordered. In the example above, the JsonDocument list contains all 5 documents but the order changes randomly from call to call. Is there any ellegant way to order the result using JavaRx capabilities or couchbase Java SDK capabilities?

The only solution i can think of is saving the results in to a HashMap and then transform the original list of ids using this HashMap into an ordered list of JsonDocuments.

Eyal
  • 1,748
  • 2
  • 17
  • 31

2 Answers2

2

Instead of flatMap, you can either use:

  • concatMap: will retain order, but actually wait for each inner GET to complete before firing the next one (could revert to sequential execution with less performance)
  • concatMapEager: will immediately subscribe inner Observables (so trigger inner GET). Maintains the order by buffering responses that arrive out of order until they can be replayed at the correct index in the sequence. Best of both worlds in terms of ordering and performance.
Simon Baslé
  • 27,105
  • 5
  • 69
  • 70
  • Is there any reason to choose concatMap over concatMapEager? the order of inner GET executions doesn't matter, but the order of the returned list should match the input list. – Eyal Aug 09 '16 at 16:29
  • ConcatMap has no buffer, which could grow quite large if eg all responses are in reverse order and you do a lot of get. That would be memory overhead. Other than that, not really AFAIK – Simon Baslé Aug 09 '16 at 17:06
0

I would use Zip operator to concat all your observables, and then once they finish add documents results into the list

   @Test
public void zipObservables() {
    Observable<String> oKey1 = Observable.just("key1").doOnNext(getDocument());
    Observable<String> oKey2 = Observable.just("key2").doOnNext(getDocument());
    Observable<String> oKey3 = Observable.just("key3").doOnNext(getDocument());
    Observable<String> oKey4 = Observable.just("key4").doOnNext(getDocument());

    List<Observable<String>> observables = Arrays.asList(oKey1,oKey2,oKey3,oKey4);
    List<Object> foundDocs = Observable.zip(observables, Arrays::asList)
            .toBlocking()
            .single();
}

private Action1<String> getDocument() {
    return id -> bucket.async().get(id);
}

You can see more Zip examples here https://github.com/politrons/reactive/blob/master/src/test/java/rx/observables/combining/ObservableZip.java

paul
  • 12,873
  • 23
  • 91
  • 153
  • It probably would work but the keys are given as a input list so the amount of keys might change, so for my specific problem it would not work – Eyal Aug 09 '16 at 14:11
  • You can pass a collection to Zip operator so you should be good with that. Check the update in my example – paul Aug 09 '16 at 15:15
  • Still, it refers to 4 keys. can you change the method to be zipObservables(List keys) and still return the results as you suggested? – Eyal Aug 09 '16 at 15:27
  • It´s an example about how you can pass a collection to the zip. Obviously the collection will be provided by you dynamically with N observables. Here in StackOverflow we give you a insight about how to do it. Please avoid copy/paste practice, it´s really bad ;) – paul Aug 09 '16 at 15:28
  • Anyway, looks like concatMap is more suitable for my needs than zip. thanks – Eyal Aug 09 '16 at 16:10