I have an HttpClient to access a REST service and now I'm implementing a soft shutdown which means that when the process is shutdown (using a simple kill command on linux) it should stop receiving new tasks and finish all the tasks it was processing.
My HttpClient has Listener registered so I can handle the REST server response.
The soft shutdown was achieved that by using a ShutdownHook but that is not exactly the problem.
When testing the solution I stumbled on this exception:
com.myproject.commons.exception.ErrorCodeException: A network error ocurred during the callback
at com.myproject.callback.client.service.CallbackLogListener.adaptCallbackException(CallbackLogListener.java:386)
at com.myproject.callback.client.service.CallbackLogListener.saveUserError(CallbackLogListener.java:279)
at com.myproject.callback.client.service.CallbackLogListener.onComplete(CallbackLogListener.java:198)
at com.myproject.callback.client.service.MultiResponseListener$8.run(MultiResponseListener.java:143)
at com.myproject.callback.client.service.MultiResponseListener$8.run(MultiResponseListener.java:140)
at com.myproject.util.ListenerCollection.foreach(ListenerCollection.java:84)
at com.myproject.callback.client.service.MultiResponseListener.onComplete(MultiResponseListener.java:140)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:185)
at org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:459)
at org.eclipse.jetty.client.HttpReceiver.abort(HttpReceiver.java:540)
at org.eclipse.jetty.client.HttpChannel.abortResponse(HttpChannel.java:129)
at org.eclipse.jetty.client.HttpChannel.abort(HttpChannel.java:122)
at org.eclipse.jetty.client.HttpExchange.abort(HttpExchange.java:257)
at com.myproject.callback.client.http.OverriddenHttpConversation.abort(OverriddenHttpConversation.java:140)
at org.eclipse.jetty.client.HttpRequest.abort(HttpRequest.java:748)
at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.abort(HttpConnectionOverHTTP.java:205)
at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.close(HttpConnectionOverHTTP.java:191)
at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.close(HttpConnectionOverHTTP.java:182)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at org.eclipse.jetty.client.AbstractConnectionPool.close(AbstractConnectionPool.java:193)
at org.eclipse.jetty.client.DuplexConnectionPool.close(DuplexConnectionPool.java:233)
at org.eclipse.jetty.client.HttpDestination.close(HttpDestination.java:372)
at org.eclipse.jetty.client.HttpClient.doStop(HttpClient.java:252)
at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
at com.myproject.callback.client.http.MyProjectHttpClient.destroy(MyProjectHttpClient.java:243)
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:262)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:972)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:979)
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1006)
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:982)
at com.myproject.MyProjectClassPathXmlApplicationContext.access$200(MyProjectClassPathXmlApplicationContext.java:10)
at com.myproject.MyProjectClassPathXmlApplicationContext$1.run(MyProjectClassPathXmlApplicationContext.java:49)
Caused by: java.nio.channels.AsynchronousCloseException: null
... 18 common frames omitted
This happens because I sent the SIGTERM signal while some of the HttpClient requests were sent to the REST server but the listener was waiting for the response and the connection was broken.
How do I prevent the HttpClient from terminating the current connections until they are all complete? (respecting a timeout probably).
Update: Apparently that is something Jetty won't support natively (https://github.com/eclipse/jetty.project/issues/1445). Would that be ok if for each destination on the HttpClient I check if its connectionPool is empty? If I get it right that would mean that there are no more open connections for this particular pool.
