0

I have a fully reactive web app that aggregates the information from two other backend-services.

Incoming request -> sends request to service A and B -> aggregates responses -> response is emitted.

pseudocode:

public Mono<ResponseEntity<List<String>>> getValues() {
     return Mono.zip(getValuesA(), getValuesB(), 
              (a, b) -> Stream.concat(a.stream(), b.stream()).collect(Collectors.toList()))
         .map(result -> ResponseEntity.ok(result));
}

public Mono<String> getValuesA() {
    return webClient.get()
             .uri(uriA)
             .retrieve()
             .bodyToMono(new ParameterizedTypeReference<>() {});
}

// getValuesB same as A, but with uriB.

Because of the high request frequency, I want to bundle requests to the backend-services. I thought using Sinks would be the right way to go. A sink is returned as mono to every requesting party. After a threshold of 10 requests has been exceeded, the request will be handled and the response will be emitted to every sink.

public Mono<ResponseEntity<List<String>>> getValues() {
     return Mono.zip(getValuesA(), getValuesB(), 
              (a, b) -> Stream.concat(a.stream(), b.stream()).collect(Collectors.toList()))
         .map(result -> ResponseEntity.ok(result));
}

public Mono<String> getValuesA() {

    Sink.One<List<String>> sink = Sinks.one();
    queue.add(sink);

    if(queue.size() > 10) {

        webClient.get()
             .uri(uriA)
             .retrieve()
             .bodyToMono(new ParameterizedTypeReference<>() {})
             .subscribe(response -> {
                 for(Sink.One<List<String>> sinkItem : queue) {
                     sink.tryEmitValue(response);
                 }
             });
    }
    
    return sink.asMono();

}

// getValuesB same as A, but with uriB.

The problem in this code is the 'subscribe' part. As soon as we're subscribing to the webclient's response, it will block the thread. This will only happen in 10% of the requests, but this is already too much for an endpoint that's being called very frequently. What can I do to 'unblock' this part. If using sinks wasn't the best choice, what could have been a better one?

PS. All pseudocode used is NOT production code. It may have many flaws and it is only meant to visualize the problem I'm facing at this moment.

Robert van der Spek
  • 1,017
  • 3
  • 16
  • 37
  • you could place this under its own `@Scheduler` annotation, that runs in a continuous loop every say second that does your `if-check` and when the que has reached 10 it will do the request and place the response in the sinks. That way the rest call is in its own "thread/loop" completly independendant from the rest calls to the service. – Toerktumlare Apr 06 '21 at 10:03
  • Excellent suggestion. I was also thinking of using kafka to get it running separately, but your suggestion is much better. I was though hoping that there would be a JavaRX-way of solving this problem. It still feels a bit like a work-around. – Robert van der Spek Apr 06 '21 at 10:07
  • I'm not sure a sink is the correct solution here, but I'm also not understanding the root problem fully - could you elaborate? Specifically, when the "10 request" threshold has been exceeded, do you merge the 10 incoming requests somehow before sending it off to the web service and then need to "unmerge" them after the request is complete, or have I got the wrong end of the stick? – Michael Berry Apr 06 '21 at 10:10
  • That's correct. I tried to keep the pseudo code as clean as possible, but yes, the url takes a parameter and returns a response for every id passed in the query. When returning I map the result to the oringal request and use its sink to emit the result. If not sink, what would be a better way of dealing this? – Robert van der Spek Apr 06 '21 at 10:13
  • 1
    So you sort of want to do a batch call. Its a sort of ”throttle control” against the underlying internal service. 10 requests come in, they are batched into a single request against the underlying internal service. Internal service returns the response, the response is the split up and pumped out the all listeners. – Toerktumlare Apr 06 '21 at 10:42
  • That's 100% correct. – Robert van der Spek Apr 06 '21 at 10:43
  • You could also do something on `doOnSubscribe` that will do the check, when 10 is reached do the the request, `doOnSubscribe` is for side effects andi believe should return immediatly. – Toerktumlare Apr 06 '21 at 10:45

1 Answers1

1

Because of the high request frequency, I want to bundle requests to the backend-services. I thought using Sinks would be the right way to go.

You shouldn't need a sink to do this at all - assuming a Flux as input, you should be able to do this in 3 steps with a standard reactive chain:

  • Buffer the input with a length of 10, which transforms your Flux<Foo> into a Flux<List<Foo>> where each element is a list of size 10 (or lower than 10 if the flux completes with fewer than 10 remaining elements);
  • Flatmap to a zipped mono which contains the original list, the "A" web service response given the list, and the "B" web service response given the list;
  • Implement a method (let's call it expand()) which takes the original list of 10 items, the A service response, and the B service response, and then splits it out into a flux of multiple items. Flatmap to this method.

The end result would be a reactive chain that looked something like:

    input.buffer(10)
        .flatMap(list -> Mono.zip(Mono.just(list), getResponseFromA(list), getResponseFromB(list))) 
        .flatMap(response -> expand(response.getT1(), response.getT2(), response.getT3()))
Michael Berry
  • 70,193
  • 21
  • 157
  • 216
  • I'm pretty new to the reactive interface, so please bare with me. My input is a String (or a List of Strings). I could create a Flux from the input, but at what point in the application flow would that be logical? – Robert van der Spek Apr 06 '21 at 11:38
  • @RobertvanderSpek I'm probably still not grasping the full context. Are we talking about something like a Kafka stream (which would take a `Flux` as the stream of messages from Kafka) or something like a REST endpoint that takes a `String` as input and outputs a `Mono` as output? Or something else entirely? – Michael Berry Apr 06 '21 at 13:08
  • The last of the two: a rest endpoint taking a List and returning a Mono. The backend services have the same signature. – Robert van der Spek Apr 06 '21 at 13:51
  • @RobertvanderSpek Cool, so can you do a `Flux.fromIterable(list)` to create your flux at the start of the request? – Michael Berry Apr 06 '21 at 15:56
  • I've been trying to get my head around this, but I am failing to grasp the concept. Maybe I've been explaining my problem the wrong way. My app gets 10 separate requests at an endpoint. These 10 requests should be bundled into 1 request to the backend. The response should then be split to respond to the 10 original requests. Is what you're describing a solution to this problem? – Robert van der Spek Apr 07 '21 at 13:09
  • Because I don't really see how flux.buffer can gather and bundle the 10 requests to one and then respond to each of them again separately. – Robert van der Spek Apr 07 '21 at 13:10