3

I'm using project reactor and I'd like to perform the following:

    @Override
    public void run(ApplicationArguments args) {
        Flux.from(KafkaReceiver.create(receiverOptions)
                        .receive()
                        .map(this::getObject)
                        .flatMap(this::iterateElasticWrites)
                        .flatMap(this::writeTheWholeObjectToS3)
        ).subscribe();
    }

    // What I'd like to do - but a non reactive code
    private Publisher<MyObj> iterateElasticWrites(MyObj message) {
        for (MyDoc file: message.getDocs()) {
            writeElasticDoc(file.getText());
        }
        return Mono.just(message);
    }

I'm struggling in finding out the equivalent of the iterateElasticWrites in Project Reactor. I'd like to perform an iteration of an object of mine (MyObj), and write each of its documents list's element into elasticsearch reactively.

ukrwine10
  • 55
  • 1
  • 6
  • `flatMapIterable()` ? So, you just do in your stream: `.flatMapIterable(myObj::getDocs)`. Then your `iterateElasticWrites()` would deal with a single `MyDoc`. – Artem Bilan Apr 18 '22 at 13:29

1 Answers1

4

In Reactor you always need to construct a reactive flow using different operators and all reactive/async code should return Mono or Flux.

Looking at your example it could look like

private Mono<MyObj> iterateElasticWrites(MyObj message) {
    return Flux.fromIterable(message.getDocs())
            .flatMap(doc -> writeElasticDoc(doc.getText()))
            .then(Mono.just(message));
}

where writeElasticDoc could be defined as

private Mono<Void> writeElasticDoc(String text) {
    ...
}
Alex
  • 4,987
  • 1
  • 8
  • 26
  • the key aspect on top of that is that you need an asynchronous non-blocking mean of writing the doc to Elastic (and later to s3). As a last resort you should switch to the boundedElasticScheduler before the first `flatMap` with `.publishOn(Schedulers.boundedElastic())` – Simon Baslé Apr 19 '22 at 09:56
  • but its considered fine returning the value with `then(Mono.just(message))`? Doesn't it create a new publisher? – ukrwine10 May 07 '22 at 13:08
  • It this example we are returning original object `Mono` but if we don't need this object downstream we could simply return`Mono`. There is nothing wrong with using `then()`. It's commonly used to execute publishers sequentially that are not dependent to each other then -> then -> then. – Alex May 07 '22 at 21:35