Combine database and network call with RxJava2

Question

I have 2 data sources: database (cache) and api and I need to combine them into one stream. I know that I can simply use concatArray or something similar but I want to achieve more complicated behavior:

Observable stream which will emit up to 2 elements.
It will subscribe to both sources in the beginning.
If api call will be fast enough (<~300ms), it will emit only data from it and complete the stream.
If api call will be slow (>~300ms), emit data from database and still wait for data from api
If api call won't succeed, emit data from database and emit an error.
If database somehow will be slower than api, it can't emit its data (stream completion solves it)

I accomplished it with the following code:

    public Observable<Entity> getEntity() {
    final CompositeDisposable disposables = new CompositeDisposable();
    return Observable.<Entity>create(emitter -> {
        final Entity[] localEntity = new Entity[1];

        //database call:
        disposables.add(database.getEntity()
                .subscribeOn(schedulers.io())
                .doOnSuccess(entity -> localEntity[0] = entity) //saving our entity because 
                                                        //apiService can emit error before 300 ms 
                .delay(300, MILLISECONDS)
                .subscribe((entity, throwable) -> {
                    if (entity != null && !emitter.isDisposed()) {
                        emitter.onNext(entity);
                    }
                }));

        //network call:
        disposables.add(apiService.getEntity()
                .subscribeOn(schedulers.io())
                .onErrorResumeNext(throwable -> {
                    return Single.<Entity>error(throwable) //we will delay error here
                            .doOnError(throwable1 -> {
                                if (localEntity[0] != null) emitter.onNext(localEntity[0]); //api error, emit localEntity
                            })
                            .delay(200, MILLISECONDS, true); //to let it emit localEntity before emitting error
                })
                .subscribe(entity -> {
                    emitter.onNext(entity); 
                    emitter.onComplete(); //we got entity from api, so we can complete the stream
                }, emitter::onError));
    })
            .doOnDispose(disposables::clear)
            .subscribeOn(schedulers.io());
}

Code is a bit clunky and I'm creating here observables inside observable, which I think is bad. But that way I have global access to emitter, which allows me to control main stream (emit data, success, error) in the way I want.

Is there better way to achieve this? I'd love to see some code examples. Thanks!

ctranxuan · Answer 1 · 2018-08-31T13:22:29.450

May be the code below could do the job. From your requirements, I assumed that the api and database deal with Single<Entity>.

private static final Object STOP = new Object();

public static void main(String[] args) {
    Database database = new Database(Single.just(new Entity("D1")));
    ApiService apiService = new ApiService(Single.just(new Entity("A1")));
    // ApiService apiService = new ApiService(Single.just(new Entity("A1")).delay(500, MILLISECONDS));
    // ApiService apiService = new ApiService(Single.error(new Exception("Error! Error!")));
    BehaviorSubject<Object> subject = BehaviorSubject.create();

    Observable.merge(
        apiService.getEntity()
                  .toObservable()
                  .doOnNext(t -> subject.onNext(STOP))
                  .doOnError(e -> subject.onNext(STOP))
                  .onErrorResumeNext(t ->
                                        Observable.concatDelayError(database.getEntity().toObservable(),
                                                                    Observable.error(t))),
        database.getEntity()
                .delay(300, MILLISECONDS)
                .toObservable()
                .takeUntil(subject)
    )
    .subscribe(System.out::println, 
               System.err::println);

    Observable.timer(1, MINUTES) // just for blocking the main thread
              .toBlocking()
              .subscribe();
}

I didn't manage to remove the use of a Subject due to the conditions "If database somehow will be slower than api, it can't emit its data" and "If api call will be slow (>~300ms), emit data from database and still wait for data from api". Otherwise, amb() operator would be a nice use.

I hope this helps.

score 1 · Answer 2 · answered Aug 31 '18 at 14:10

Another solution could be this one (no subject):

public static void main(String[] args) throws InterruptedException {
    Database database = new Database(Single.just(new Entity("D1")));
    ApiService apiService = new ApiService(Single.just(new Entity("A1")));
    // ApiService apiService = new ApiService(Single.just(new Entity("A1")).delay(400, MILLISECONDS));
    // ApiService apiService = new ApiService(Single.error(new Exception("Error! Error!")));

    database.getEntity()
            .toObservable()
            .groupJoin(apiService.getEntity()
                                 .toObservable()
                                 .onErrorResumeNext(
                                    err -> Observable.concatDelayError(database.getEntity().toObservable(),
                                                                       Observable.error(err))),
                       dbDuration -> Observable.timer(300, MILLISECONDS),
                       apiDuration -> Observable.never(),
                       (db, api) -> api.switchIfEmpty(Observable.just(db)))
            .flatMap(o -> o)
            .subscribe(System.out::println,
                       Throwable::printStackTrace,
                       () -> System.out.println("It's the end!"));

    Observable.timer(1, MINUTES) // just for blocking the main thread
              .toBlocking()
              .subscribe();
}

If nothing is emitted from the API service within the 300 ms (dbDuration -> timer(300, MILLISECONDS)), then the entity from the database is emitted (api.switchIfEmpty(db)).

If api emits something within the 300 ms, then the emits only its Entity (api.switchIfEmpty(.)).

That seems to work as you wish too...

score 1 · Answer 3 · answered Oct 17 '18 at 09:24

Another nicer solution:

public static void main(String[] args) throws InterruptedException {
    Database database = new Database(Single.just(new Entity("D1")));
    ApiService apiService = new ApiService(Single.just(new Entity("A1")));
    // ApiService apiService = new ApiService(Single.just(new Entity("A1")).delay(400, MILLISECONDS));
    // ApiService apiService = new ApiService(Single.error(new Exception("Error! Error!")));

    Observable<Entity> apiServiceWithDbAsBackup =
            apiService.getEntity()
                      .toObservable()
                      .onErrorResumeNext(err -> 
                            Observable.concatDelayError(database.getEntity().toObservable(), Observable.error(err)));

    Observable.amb(database.getEntity()
                           .toObservable()
                           .delay(300, MILLISECONDS)
                           .concatWith(apiServiceWithDbAsBackup),
                   apiServiceWithDbAsBackup)
              .subscribe(System.out::println,
                         Throwable::printStackTrace,
                         () -> System.out.println("It's the end!"));

We use amb() with a delay to for database observable to take the first one which will emits. In case of error of the api service, we emit the item from the database. That seems to work as you wish too...

Combine database and network call with RxJava2

3 Answers3