Specify server request timeout in Spring Boot & Spring WebFlux (Netty)

Question

We are using Spring Boot in 2.4.2 with Spring WebFlux.

I want the Spring Boot application to terminate all requests to the application that take longer than say 3 seconds to process.

There is server.netty.connection-timeout, but that doesn't seem to do the trick.

Is there a way to specify such a server request timeout?

Answer 1

TL;DR:

Netty has no request timeout*. Add this WebFilter to set a request-timeout of 3 seconds using the reactor timeout on every request (here in kotlin, but in Java it works accordingly):

@Component
class RequestTimeoutWebFilter : WebFilter {
  override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
    return chain
      .filter(exchange)
      .timeout(Duration.ofSeconds(3))
  }
}

* at least I could not find any information in the netty docs.

Detailed answer

The connection-timeout does not refer to the duration that a request is allowed to take for processing, but it refers to the time it takes for establishing the connection.

First, I could not find any spring configuration option that allows setting the request timeout for netty. Then I went through the netty documentation to find out that there is no concept of request timeouts on the http server (only on the http client).

Wondering about why such important feature would not be supported, I remembered that we often cannot apply the same concepts as in blocking servers for good reasons. Next, I remembered, that in the reactive world we do not directly implement the handling of the request, but how the handling is assembled - i.e. we hold a Mono<Void> that will handle the request. Hence, we can just look at reactor and how to timeout a Mono, which is very easy:

Mono.create(...)
  .timeout(Duration.ofSeconds(3))

Next, we just need to figure out, how to apply this to all requests. This is easy as well, because we can just use a WebFilter to intercept all requests to apply our timeout (here in kotlin, but in Java it works accoringly):

@Component
class RequestTimeoutWebFilter : WebFilter {
  override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
    return chain
      .filter(exchange)
      .timeout(Duration.ofSeconds(3))
  }
}

This effectively cancels a request within the set timeout with the following error:

2022-10-21 00:08:00.981 ERROR 6289 --- [     parallel-4] a.w.r.e.AbstractErrorWebExceptionHandler : [59dfa990-7]  500 Server Error for HTTP GET "/some/route"

java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 3000ms in 'source(MonoDefer)' (and no fallback has been configured)
    at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.handleTimeout(FluxTimeout.java:295) ~[reactor-core-3.4.22.jar:3.4.22]

More tips and hints

• To make the timeout configurable, we can use a custom config variable instead of the hard-coded duration.
• To custimize the 500 status code we can either change the exception by providing a fallback to the timeout as 2nd argument and handle that exception in a controller advice - or we can just use reactors onErrorReturn.
• The documentation for WebFilter actually states that they should be used to implement timeouts:

  Contract for interception-style, chained processing of Web requests that may be used to implement cross-cutting, application-agnostic requirements such as security, timeouts, and others.

Still I think it is expected that spring provides such implementation out-of-the box that can be easily configured. Maybe we oversaw that it is there, but then I would argue it is too hard to find. ^^

Alternative solution path

As an alternative, we could use circuit breakers. However, those are implemented on the readers side and conceptually are used to protect the reading side against failure of the downstream - rather than protecting the internal processing within the downstream from running too long. They can only be applied to mimic a request timeout when applying them in a dedicated server (e.g. a spring cloud gateway server) that sits between the actual client and the actual service. When using Resilience4j as implementation, you can use TimeLimiter to achieve it.

Answer 2

I was also facing the same issue i.e. even after configuring server.netty.connection-timeout request would get canceled. So, after some debugging found that timeout was getting set to '30000' by AsyncContext.

So, I configured the following property spring.mvc.async.request-timeout which change the timeout being set in AsyncContext and the request stopped getting canceled.