Buffering capacity 2097152 exceeded from jetty when response is large

Question

I have a third-party application that uses the jetty library as HttpClient to handle webservice calls. I have the following jetty libs:

jetty-client-9.4.28.v20200408.jar
jetty-continuation-9.4.28.v20200408.jar
jetty-http-9.4.28.v20200408.jar
jetty-io-9.4.28.v20200408.jar
jetty-jndi-9.4.28.v20200408.jar
jetty-rewrite-9.4.28.v20200408.jar
jetty-security-9.4.28.v20200408.jar
jetty-server-9.4.28.v20200408.jar
jetty-servlet-9.4.28.v20200408.jar
jetty-servlets-9.4.28.v20200408.jar
jetty-util-9.4.28.v20200408.jar
[EDIT]
jersey-bean-validation-2.30.1.jar
jersey-client-2.30.1.jar
jersey-common-2.30.1.jar
jersey-container-jetty-http-2.30.1.jar
jersey-container-servlet-2.30.1.jar
jersey-container-servlet-core-2.30.1.jar
jersey-entity-filtering-2.30.1.jar
jersey-hk2-2.30.1.jar
jersey-jetty-connector-2.30.1.jar
jersey-media-jaxb-2.30.1.jar
jersey-media-json-jackson-2.30.1.jar
jersey-media-multipart-2.30.1.jar
jersey-server-2.30.1.jar

It turns out that the third-party code is not written properly, and the jetty in this version has, as far as I know, the size of the response buffer set by default to 2MB. Probably because when the response from the webservice exceeds 2MB, I have errors like:

java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Buffering capacity 2097152 exceeded: ==> javax.ws.rs.ProcessingException - java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Buffering capacity 2097152 exceeded
      at: JettyConnector.apply(JettyConnector.java:269)
  - Cause: java.util.concurrent.ExecutionException - java.lang.IllegalArgumentException: Buffering capacity 2097152 exceeded
      at: FutureResponseListener.getResult(FutureResponseListener.java:118)
  - Cause: java.lang.IllegalArgumentException - Buffering capacity 2097152 exceeded
      at: BufferingResponseListener.onContent(BufferingResponseListener.java:124)

My guess is that this is about a buffer for FutureResponseListener as it is written in the thread Does Jetty's httpClient.setResponseBufferSize() method do anything? .

I want to be able to change / increase this buffer without changing the third-party code that uses jetty. I heard that I can set the buffer size using the jetty.xml configuration file, maybe something like this:

    <New id = "httpConfig" class = "org.eclipse.jetty.server.HttpConfiguration">
      <Set name = "????"> <Property name = "????" default = "8388608" /> </Set>
    </New>

Am I right ? Can it be done in my case and is it enough to put a properly prepared jetty.xml file on the classpath?

[EDIT] ... or maybe there is a problem with jersey-client (JettyConnector object) which uses jetty ...

Thanks for help,

koli

score 1 · Answer 1 · answered Jun 16 '21 at 21:45

I also got the same error when trying to do a GET request on an API which sent me a streaming output which was greater than buffer capacity.

I figured out that using an InputStreamListener solved this issue. ( https://www.eclipse.org/jetty/javadoc/jetty-9/org/eclipse/jetty/client/util/InputStreamResponseListener.html)

My code snippet below:

HttpClient httpClient = new HttpClient;
httpClient.start()
URI uri = UriBuilder.fromUri(URI.create(ENDPOINT)).path("v1/testPath").queryParam("testQueryParam","testValue").build();
InputStreamResponseListener listener = new InputStreamResponseListener();
httpClient.newRequest(uri).method(HttpMethod.GET).headers(httpFields -> httpFields.add("Authorization","Bearer "+token)).send(listener);
Response response =  listener.get(20, TimeUnit.SECONDS);
if(response.getStatus() == 200){
     //doSomething
    }
httpClient.stop();

Buffering capacity 2097152 exceeded from jetty when response is large

1 Answers1