Rx Observables: emit additional item for each original item, reduce them to another type, consume

Question

I am having problems doing the following using Couchbase Java client 2.2.2 and Rx Observables 1.0.15:

I have a list of strings which are document names
Along with each original document for a document name I would like to load another document (deduced from the original document name) so I would get a pair of documents. If any of those two documents do not exist, do not use this pair any more.
If the pair is valid (i.e. both documents exist) then use both documents to create a custom object from them
combine those transformed items into a list

What I have come up with so far looks really mean:

List<E> resultList = new ArrayList<>();

Observable
    .from(originalDocumentNames)
    .flatmap(key -> {
        Observable firstDocument = bucket.async().get(key);
        Observable secondDocument = bucket.async().get(getSecondKeyNameFrom(key));
        return Observable.merge(firstDocument, secondDocument);
    })
    .reduce((jsonDocument1, jsonDocument2) -> {
        if (jsonDocument1 == null || jsonDocument2 == null) {
            return null;
        }
        resultList.add(createCustomObject(jsonDocument1, jsonDocument2);
        return null;
    })
    .filter(Objects.nonNull)
    .singleOrDefault(null)
    .subscribe(new Subscriber<E>() {
        public void onComplete() {
            //use resultList in a callback function
        }
    });

This does not work. I do not know where, but I think I am using Observable.merge the wrong way. Also I think I am approaching the whole problem the wrong way.

So the main questions it seems are:

how do I emit an additional item to an Observable stream?
how can I reduce two items into an item of another type? (reduce(T, T, T) does not allow that)
am I taking it on wrong?

I've not much experience with RxJava — work in the Scala world mostly, but the normal way to expand elements is exactly what you are doing: you either `flatMap`, expanding or shrinking the elements as you wish, or use `flatMapIterable` if you can provide the expansion immediately. — alf, Dec 13 '15 at 22:36
What you may want to do (and what SO guidelines tell you, anyway) is to reduce the example, removing all the details (what's `bucket`? Why would anyone care?) until you have a minimal self-containing example reproducing the problem. It well may help you solve the problem as you go. — alf, Dec 13 '15 at 22:38
You might want to play with the zip() function instead of merge() and reduce(). http://reactivex.io/documentation/operators/zip.html — John Scattergood, Dec 14 '15 at 04:15
@alf It is already most minimal. And "bucket" in the context if a Couchbase application is pretty clear. — Sebastian, Dec 14 '15 at 10:14
@JohnScattergood When using zip() the processed elements overlap. I want to emit two items, process these two into a new one and not use any of the original items in the next processing step. — Sebastian, Dec 14 '15 at 10:15

score 4 · Accepted Answer · answered Dec 14 '15 at 11:08

You could use zip inside the flatmap. Zip will emit as many items as the Observable with the fewest items. So if one of the documents is missing, its sequence will be empty and zip will skip it.

Observable
.from(originalDocumentNames)
.flatmap(key -> {
    //the stream of 0-1 original document
    Observable firstDocument = bucket.async().get(key);
    //the stream of 0-1 associated document
    Observable secondDocument = bucket.async().get(getSecondKeyNameFrom(key));

    //using zip and the createCustomObject method reference as a zip function to combine pairs of documents
    return Observable.zip(firstDocument, secondDocument, this::createCustomObject);
})
.toList() //let RxJava aggregate into a List
.subscribe(
    //the "callback" function, onNext will be called only once with toList
    list -> doSomething(list), 
    //always try to define onError (best practice)
    e -> processErrors(e)
);

Thx, that does it. Didn't think of a zip inside a flatmap. – Sebastian Dec 14 '15 at 11:32 — Sebastian, Dec 14 '15 at 11:32

score 1 · Answer 2 · answered Dec 14 '15 at 11:20

There's several issue in this code :

side effect, the reduce operation is adding to a list outside the Observable chain, that's wrong. The reduce should either return the list or don't exists at all as Rx has a toList operation. Also because of the reduce operation that returns null the next operations have to handle that ; this is rather inelegant.
merge operation is wrong, you should instead zip in the flatmap and build the pair/aggregate.
Optional point : the flatmap operation does not handle if either get operation will return multiple items (maybe that's de facto the case with couchbase)

Note I don't have an IDE so no code for now. But in my point replacing merge by zip and removing reduce certainly should help.

Rx Observables: emit additional item for each original item, reduce them to another type, consume

2 Answers2