RxJava concurrency with multiple subscribers and events

Question

I'm looking for a way to attach multiple subscribers to an RxJava Observable stream, with each subscriber processing emitted events asynchronously.

I first tried using .flatMap() but that didn't seem to work on any subsequent subscribers. All subscribers were processing events on the same thread.

.flatMap(s -> Observable.just(s).subscribeOn(Schedulers.newThread()))

What ended up working was consuming each event in a new thread by creating a new Observable each time:

Observable.from(Arrays.asList(new String[]{"1", "2", "3"}))
            .subscribe(j -> {
                Observable.just(j)
                        .subscribeOn(Schedulers.newThread())
                        .subscribe(i -> {
                            try {
                                Thread.sleep(ThreadLocalRandom.current().nextInt(100, 500));
                            } catch (InterruptedException e) {
                                e.printStackTrace();
                            }
                            System.out.println("s1=>" + Thread.currentThread().getName() + "=>" + i);
                        });
            });

Output:

s1=>RxNewThreadScheduler-1=>1
s1=>RxNewThreadScheduler-2=>2
s1=>RxNewThreadScheduler-3=>3

And the end result with multiple subscribers:

ConnectableObservable<String> e = Observable.from(Arrays.asList(new String[]{"1", "2", "3"}))
            .publish();

    e.subscribe(j -> {
        Observable.just(j)
                .subscribeOn(Schedulers.newThread())
                .subscribe(i -> {
                    try {
                        Thread.sleep(ThreadLocalRandom.current().nextInt(100, 500));
                    } catch (InterruptedException e1) {
                        e1.printStackTrace();
                    }
                    System.out.println("s1=>" + Thread.currentThread().getName() + "=>" + i);
                });
    });

    e.subscribe(j -> {
        Observable.just(j)
                .subscribeOn(Schedulers.newThread())
                .subscribe(i -> {
                    try {
                        Thread.sleep(ThreadLocalRandom.current().nextInt(100, 500));
                    } catch (InterruptedException e1) {
                        e1.printStackTrace();
                    }
                    System.out.println("s2=>" + Thread.currentThread().getName() + "=>" + i);
                });
    });

    e.connect();

Output:

s2=>RxNewThreadScheduler-4=>2
s1=>RxNewThreadScheduler-1=>1
s1=>RxNewThreadScheduler-3=>2
s2=>RxNewThreadScheduler-6=>3
s2=>RxNewThreadScheduler-2=>1
s1=>RxNewThreadScheduler-5=>3

However, this seems a little clunky. Is there a more elegant solution or is RxJava just not a good use case for this?

score 0 · Answer 1 · answered Nov 06 '16 at 11:07

0

Use .flatMap(s -> Observable.just(s).observeOn(Schedulers.newThread())....)

answered Nov 06 '16 at 11:07

Tassos Bassoukos

16,017
2
36
40

I originally tried this (both .observeOn and .subscribeOn), but whenever I introduced a computation in my subscriber that took more than a few milliseconds e.g. Thread.sleep(100), everything ended up being processed on the same thread again. – Nov 06 '16 at 18:25

score 0 · Answer 2 · answered Nov 06 '16 at 11:08

if I understood the rx-contract correctly, you are trying to do something, which is against it.

Lets have a look at the contract

The contract of an RxJava Observable is that events ( onNext() , onCompleted() , onEr ror() ) can never be emitted concurrently. In other words, a single Observable stream must always be serialized and thread-safe. Each event can be emitted from a different thread, as long as the emissions are not concurrent. This means no inter‐ leaving or simultaneous execution of onNext() . If onNext() is still being executed on one thread, another thread cannot begin invoking it again (interleaving). --Tomasz Nurkiewicz in Reactive Programming with RxJava

In my opinion you are trying to break the contract by using a nested subscription in the outer subscription. The onNext call to the subscriber is not serialized anymore.

Why not move the "async"-workload from the subscriber to a flatMap-operator and subscribe to the new observable:

    ConnectableObservable<String> stringObservable = Observable.from(Arrays.asList(new String[]{"1", "2", "3"}))
            .flatMap(s -> {
                return Observable.just(s).subscribeOn(Schedulers.computation());
            })
            .publish();

    stringObservable
            .flatMap(s -> {
                // do More asyncStuff depending on subscription
                return Observable.just(s).subscribeOn(Schedulers.newThread());
            })
            .subscribe(s -> {
                // use result here
            });

    stringObservable
            .subscribe(s -> {
                // use immediate result here.
            });

    stringObservable.connect();

I believe the contract applies to the Observable and not subscribers. I'm not trying to create a single Observable stream with concurrent events. But rather, I'm trying to have multiple unrelated subscribers to each event. Since they're unrelated, I'd like to process them in different threads to take advantage of parallelization. In my current use case, the subscriber makes a series of network calls which do not need any future processing, so I'm not sure what benefit would be gained by moving that to a .flatMap() operator. — , Nov 06 '16 at 18:17
"I'm not trying to create a single Observable stream with concurrent events." Actually you are trying to do exactly that. Due to the publish/ connect you only have one stream which will call onNext from subscriptions. The onNext call is serialized. Lets have a look at the Subscriber interface: void onNext(String s). You should create a new observable for each subscription, which will invoke your action in flatMap and subscriobe on and subscribe to that specific one. — Sergej Isbrecht, Nov 06 '16 at 19:48

Praveer Gupta · Answer 3 · 2016-11-07T02:11:32.523

flatMap along with doOnNext on the Observable inside the flatMap will result in the same output as yours.

onNext() is always called in a sequential manner hence using doOnNext after the flatMap will also not work for you. Due to the same reason writing the action inside the final subscribe didn't work in your case.

The below code is written using RxJava2. In version 1 of RxJava you will have to add the try-catch block around Thread.sleep.

ConnectableObservable<String> e = Observable.just("1", "2", "3").publish();

e.flatMap(
      s -> Observable.just(s)
          .subscribeOn(Schedulers.newThread())
          .doOnNext(i -> {  // <<<<<<
              Thread.sleep(ThreadLocalRandom.current().nextInt(100, 500));
              System.out.println("s1=>" + Thread.currentThread().getName() + "=>" + i);
          }))
  .subscribe();

e.flatMap(
      s -> Observable.just(s)
          .subscribeOn(Schedulers.newThread())
          .doOnNext(i -> {  // <<<<<<
              Thread.sleep(ThreadLocalRandom.current().nextInt(100, 500));
              System.out.println("s2=>" + Thread.currentThread().getName() + "=>" + i);
          }))
  .subscribe();

e.connect();

score 0 · Answer 4 · answered Apr 19 '19 at 11:07

You can achieve it with Flowable and parallel:

        Flowable.fromIterable(Arrays.asList("1", "2", "3"))
                .parallel(3)
                .runOn(Schedulers.newThread())
                .map(item -> {
                    try {
                        Thread.sleep(ThreadLocalRandom.current().nextInt(100, 500));
                    } catch (InterruptedException e1) {
                        e1.printStackTrace();
                    }
                    System.out.println("s1=>" + Thread.currentThread().getName() + "=>" + item);

                    return Completable.complete();
                })
        .sequential().subscribe();

RxJava concurrency with multiple subscribers and events

4 Answers4

Linked