HttpClient asyncRequest and exponential backoff

Question

I need to implement an exponential backoff on a request that might fail. However, it's implemented as an async request. Had this been done synchronously, I'd have a better idea on where to put the delay. Roughly, I'm thinking it'd work something like this:

// These would be configurable in application.yml
currentAttempt = 0;
maxAttempts = 3;
timeoutGrowth = 2;
currentDelayTime = 5ms;
repeatNeeded = false;
while(repeatNeeded && currentAttempt < maxAttempts) {
    httpStatusCode = makeRequest(someService)
    if(httpStatusCode == 503) {
        repeatNeeded=true;
        currentAttempt++;
        currentDelayTime*=timeoutGrowthRate;
        sleep(currentDelayTime)
    }
}

However, with an async call, the caller to the function is given the time back to do something else until the Future is has something. Do I code the backoff within the getObservations() method below, or do I code this in the caller of that getObservations() method? Below is the call as it currently is:

public CompletableFuture getObservations(String text, Map<String, Object> bodyParams) throws URISyntaxException { URI uri = getUri(text); HttpRequest request = getRequest(uri, text, bodyParams); Map<String, String> contextMap = Optional.ofNullable(MDC.getCopyOfContextMap()).orElse(Collections.emptyMap());

    Instant startTime = Instant.now();

    return httpClient.sendAsync(request, BodyHandlers.ofString())
            .exceptionally(ex -> {
                throw new ExternalToolException(externalServiceConfig.getName(), ex);
            })
            .thenApply(response -> {
                long toolRequestDurationMillis = ChronoUnit.MILLIS.between(startTime, Instant.now());
                if (HttpStatus.valueOf(response.statusCode()).is2xxSuccessful()) {
                    ToolResponse toolResponse = processResponse(response, toolRequestDurationMillis);
                    logToolResponse(toolResponse);
                    return toolResponse;
                }

                log.error("{} returned non-200 response code: {}", externalServiceConfig.getName(), response.statusCode());
                throw new ExternalToolException(externalServiceConfig.getName(), response.statusCode());
            });

}

if you could consider using reactive java, it has very powerful API including retries For example,`.retryWhen(Retry.backoff(3, Duration.ofSeconds(2)));` You could use`WebClient` that is non-blocking, reactive client or wrap `CompletableFeature` using `Mono.fromFuture` — Alex, Mar 11 '22 at 03:00
@Alex This looks like a great answer. Just to be clear, did you mean CompletableFuture? I wonder how this would work if the caller were say an Angular server, and not an internal Java code. That is, how would it return for a little bit, then come back when the real result was ready. Anyway, I'd like to give you credit for your answer, would you please post as such? — Woodsman, Mar 11 '22 at 18:06
yeah `CompletableFuture` ;-). non-blocking means that all I/O operations are async and event-driven (think event loop) small number of threads to scale. This model is similar to node.js. There is no difference for client that will get results when ready but server-side processing is more efficient. You can find some examples in answer. — Alex, Mar 11 '22 at 23:27

score 1 · Accepted Answer · answered Mar 11 '22 at 23:27

If you could consider using reactive java that has very powerful API including retries. For example,

request()
  .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));

there are more options like retries for the specific exceptions only or defining max backoff

request()
  .retryWhen(Retry.backoff(3, Duration.ofSeconds(2)));
  .maxBackoff(5)
  .filter(throwable -> isRetryableError(throwable))

You could use WebClient that is a non-blocking client exposing a fluent, reactive API over underlying HTTP client libraries such as Reactor Netty

webClient.get()
        .uri("/api")
        .retrieve()
        .bodyToMono(String.class)
        .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));

if for some reason, you still want to use HttpClient you can wrap CompletableFuture

Mono.fromFuture(httpClient.sendAsync(request, BodyHandlers.ofString()))
  .retryWhen(Retry.backoff(3, Duration.ofSeconds(1)));

I forgot to ask which package has the reactive client. Plus one for giving the maven dependency and the import statement(s). — Woodsman, Mar 15 '22 at 21:09
WebClient is in `org.springframework.boot:spring-webflux`. if you need just api - `io.projectreactor:reactor-core` — Alex, Mar 15 '22 at 21:21
For sping boot - `org.springframework.boot:spring-boot-starter-webflux` — Alex, Mar 15 '22 at 21:23

HttpClient asyncRequest and exponential backoff

1 Answers1