1

Question:

How do you configure ring-jetty-adapter to limit the number of concurrent worker threads? I'm using embedded jetty here, not creating a WAR file or anything like that.

Context

I only have 20 connections in my database connection pool, and all requests need to do database queries. Currently, when the server load gets to high, say 40 concurrent requests continually, 20 of them will be blocked waiting for the DB. Then the queue will keep building up, and the wait will just spike out of control (Thread starvation).

The :max-threads parameter does not do what I want, as it only limits the size of jettys internal thread pool, which is used for accept and selector threads, not just worker threads.

After some research I think what I need to use is the jetty QoS filter but I can't figure out how to translate a web.xml configuration to my clojure ring app.

rutchkiwi
  • 427
  • 3
  • 14

1 Answers1

1

First, you cannot control or limit behavior by adjusting threading configuration.

The old school model of 1 thread per request is not valid on modern containers, especially so on Jetty, which is 100% async internally.

A single request can use 1...n threads through that request's lifetime. The behavior of threading in Jetty is influenced by your technology choices (eg: os, jvm, network protocols, etc), and API choices, and even how stressed your server is.

With that out of the way, your desired solution should instead focus on limiting the number of Requests that can be used by a specific server resource endpoint concurrently.

The way that's done is by limiting the number of active requests that can concurrently access a server resource endpoint.

This is accomplished by tracking the number of requests to that specific resource endpoint and then suspending requests that exceed a configured maximum, resuming suspended requests when the active count falls below the configured maximum, and also timing out requests that sit in the suspended state for too long.

This feature set is provided for you in the Jetty QoSFilter.

You can use the Jetty QoSFilter for anything in Jetty that is based on the Jetty ServletContextHandler (which includes the WebAppContext).

See: https://www.eclipse.org/jetty/documentation/jetty-9/index.html#qos-filter

FilterHolder qosConfig = servletContextHandler.addFilter(QoSFilter.class,
    "/api/backend/*", 
    EnumSet.of(DispatcherType.REQUEST));
qosConfig.setInitParameter("maxRequests", "10");
qosConfig.setInitParameter("waitMs", "50");
qosConfig.setInitParameter("suspendMs", "-1");
Joakim Erdfelt
  • 46,896
  • 7
  • 86
  • 136
  • thanks! In my case as I'm using ring-jetty-adapter, I only have access to the org.eclipse.jetty.server.Server object which is already configured but not yet started. How can I add the filter on that object? – rutchkiwi Jun 18 '21 at 18:37
  • You need access to the `ServletContextHandler` or `WebAppContext` to do that (and you need to do that before the `Server.start()` is run. Walk the tree of Handlers from `Server.getHandler()` and see what you have. (Optionally, dump the entire `Server.dump()` post-start and see what the tree looks like) – Joakim Erdfelt Jun 18 '21 at 18:50
  • I don't have a ServletContextHandler or a WebAppContext.. Ring-jetty-adapter just creates a custom implementation of AbstractHandler. How would I solve this in that case? – rutchkiwi Jun 23 '21 at 15:29
  • You have no ServletContext, hence no AsyncContext, as a result you have no QoSFilter support. There's no support built out of the box in Jetty for your scenario. I really have no idea, you can't even use the implementation in QoSFilter in your own custom code, as it relies on the existence of AsyncContext and the machinery built into the ServletContext to function. – Joakim Erdfelt Jun 23 '21 at 17:59
  • Ah thank you, I understand. My limited understanding of jetty is really showing, but I've read about servlets wrapping things. Is it possible to create a ServletContextHandler wrapping another handler somehow? – rutchkiwi Jun 24 '21 at 19:10