How to merge two sorted Observables into a single sorted Observable?

Question

Given:

Integer[] arr1 = {1, 5, 9, 17};
Integer[] arr2 = {1, 2, 3, 6, 7, 12, 15};
Observable<Integer> o1 = Observable.from(arr1);
Observable<Integer> o2 = Observable.from(arr2);

How to get an Observable that contains 1, 1, 2, 3, 5, 6, 7, 9, 12, 15, 17?

Answer 1

Edit: Please see the_joric's comment if you're going to use this. There is an edge case that isn't handled, I don't see a quick way to fix it, and so I don't have time to fix it right now.

Here's a solution in C#, since you have the system.reactive tag.

static IObservable<int> MergeSorted(IObservable<int> a, IObservable<int> b)
{
    var source = Observable.Merge(
        a.Select(x => Tuple.Create('a', x)),
        b.Select(y => Tuple.Create('b', y)));
    return source.Publish(o =>
    {
        var published_a = o.Where(t => t.Item1 == 'a').Select(t => t.Item2);
        var published_b = o.Where(t => t.Item1 == 'b').Select(t => t.Item2);
        return Observable.Merge(
            published_a.Delay(x => published_b.FirstOrDefaultAsync(y => x <= y)),
            published_b.Delay(y => published_a.FirstOrDefaultAsync(x => y <= x)));
    });
}

The idea is summarized as follows.

• When a emits the value x, we delay it until b emits a value y such that x <= y.

• When b emits the value y, we delay it until a emits a value x such that y <= x.

If you only had hot observables, you could do the following. But the following would not work if there were any cold observables in the mix. I would advise always using the version that works for both hot and cold observables.

static IObservable<int> MergeSortedHot(IObservable<int> a, IObservable<int> b)
{
    return Observable.Merge(
        a.Delay(x => b.FirstOrDefaultAsync(y => x <= y)),
        b.Delay(y => a.FirstOrDefaultAsync(x => y <= x)));
}

Answer 2

You can merge, sort and flatten the sequences, but it will have a significant overhead:

o1.mergeWith(o2).toSortedList().flatMapIterable(v -> v).subscribe(...)

or

o1.concatWith(o2).toSortedList().flatMapIterable(v -> v).subscribe(...)

Otherwise, you need to write a fairly complicated operator.

Edit 04/06/2015:

Here is an operator that does this sorted-merge more efficiently.

Answer 3

I was also looking for a merge sort solution that supports a backpressure along and could not find it. So decided to implement it on my own loosely based on the existing zip operator.

Similarly to zip, the sorted merge operator collects an item from each source observable first, but then puts them into a priority queue, from which emits them one by one according to their natural order or the specified comparator.

You can grab it from GitHub as a ready to use library or just copy/paste the code:

https://github.com/ybayk/rxjava-recipes

See unit tests for usage.

Answer 4

This was discussed some time ago on the RxJava mailing list, you'll find some links to possible solutions in that thread.

Answer 5

What about just merge and sort?

@Test
public void testMergeChains() {
    Observable.merge(Observable.from(Arrays.asList(1, 2, 13, 11, 5)), Observable.from(Arrays.asList(10, 4, 12, 3, 14, 15)))
              .collect(ArrayList<Integer>::new, ArrayList::add)
            .doOnNext(Collections::sort)
            .subscribe(System.out::println);

}

You can see more examples here

https://github.com/politrons/reactive

Answer 6

My solution using custom transformer writen in Kotlin:

Transformer:

fun <T> orderedMerge(f2: Flowable<T>, c: Comparator<T>) = FlowableTransformer<T, T> { f1 ->
    val f1Iterator = f1.blockingIterable(1).iterator()
    val f2Iterator = f2.blockingIterable(1).iterator()
    Flowable.generate(
            Callable { null as T? to null as T? },
            BiFunction { (lastF1: T?, lastF2: T?), emitter: Emitter<T> ->
                when {
                    lastF1 != null && f2Iterator.hasNext() -> {
                        val nextF2 = f2Iterator.next()
                        if (c.compare(lastF1, nextF2) <= 0) {
                            emitter.onNext(lastF1)
                            null to nextF2
                        } else {
                            emitter.onNext(nextF2)
                            lastF1 to null
                        }
                    }
                    lastF1 != null -> {
                        emitter.onNext(lastF1)
                        null to null
                    }
                    lastF2 != null && f1Iterator.hasNext() -> {
                        val nextF1 = f1Iterator.next()
                        if (c.compare(nextF1, lastF2) <= 0) {
                            emitter.onNext(nextF1)
                            null to lastF2
                        } else {
                            emitter.onNext(lastF2)
                            nextF1 to null
                        }
                    }
                    lastF2 != null -> {
                        emitter.onNext(lastF2)
                        null to null
                    }
                    f1Iterator.hasNext() && f2Iterator.hasNext() -> {
                        val nextF1 = f1Iterator.next()
                        val nextF2 = f2Iterator.next()
                        if (c.compare(nextF1, nextF2) <= 0) {
                            emitter.onNext(nextF1)
                            null to nextF2
                        } else {
                            emitter.onNext(nextF2)
                            nextF1 to null
                        }
                    }
                    f1Iterator.hasNext() -> {
                        val nextF1 = f1Iterator.next()
                        emitter.onNext(nextF1)
                        null to null
                    }
                    f2Iterator.hasNext() -> {
                        val nextF2 = f2Iterator.next()
                        emitter.onNext(nextF2)
                        null to null
                    }
                    else -> {
                        emitter.onComplete()
                        null to null
                    }
                }
            })
}

Usage:

val f1 = listOf(1, 2, 3, 7, 10, 11, 11, 20).toFlowable()
val f2 = listOf(3, 4, 5, 8, 15, 16).toFlowable()

val ff = f1.compose(orderedMerge(f2, Comparator.naturalOrder()))