I am using Java 11's HttpClient
. The client hangs intermittently and I can't figure out why -- I don't know what's causing the hang and I don't know how to reproduce it.
The code to construct the client and requests are in separate locations, so this summary shows how they are constructed:
// excerpt 1: constructing the client
private HttpClient httpClient = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(5))
.build();
// excerpt 2: constructing the request
HttpRequest req = HttpRequest.newBuilder()
.GET()
.uri(...)
.header(...)
.header(...)
.timeout(Duration.ofSeconds(5))
.build();
// excerpt 3: sending the request
HttpResponse<String> resp = httpClient.send(req, BodyHandlers.ofString());
Initially, all I knew was that my service was hanging -- I didn't know HttpClient
was involved. I then threw a watchdog timer in a background thread that held a (weak) reference to the main worker thread so that if the timer expired, I could get a stack trace of what the main worker thread was doing. That gave me this stack trace (expired after five minutes):
j.u.c.TimeoutException: null // I crammed the thread's stack trace into an exception to play nice with Splunk
at j.i.m.Unsafe.park(Unknown Source)
at j.u.c.l.LockSupport.park(Unknown Source)
at j.u.c.CompletableFuture$Signaller.block(Unknown Source)
at j.u.c.ForkJoinPool.managedBlock(Unknown Source)
at j.u.c.CompletableFuture.waitingGet(Unknown Source)
at j.u.c.CompletableFuture.get(Unknown Source)
at j.i.n.h.HttpClientImpl.send(Unknown Source)
at j.i.n.h.HttpClientFacade.send(Unknown Source)
at c.a.x.d.c.MyClientClass.sendRequest(MyClientClass.java:163)
at ...
I have since seen a few of these timeouts occur, in different sections of code that all use HttpClient
, all producing the same stack trace.
I know there have been HttpClient
bugs dating back to Java 9 that can cause hangs, but we're running on a Java 11 build from last month (IIRC) and all the bugs I've read were fixed long before that.
Any suggestions on what might be causing this, or how to reproduce it or debug it further?