1

I have two services A and B where Service A is trying to get the json data from the Service B. When data is less for example 10KB the call ends without any exception. But when the data is around 100MB it gives an error java.net.SocketException: Unexpected end of file from server on Service A and we see Exception Message:java.io.IOException: Broken pipe in Service B.

We are using Spring boot 2.2.6, Java8 and RestTemplate for communication. Below is the stackTrace for Service A.

java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:862)
java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:689)
java.base/sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:859)
java.base/sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:689)
java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1615)
java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1520)
java.base/java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:527)
java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:334)
org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:82)
org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:109)
org.springframework.boot.actuate.metrics.web.client.MetricsClientHttpRequestInterceptor.intercept(MetricsClientHttpRequestInterceptor.java:93)
org.springframework.http.client.InterceptingClientHttpRequest$InterceptingRequestExecution.execute(InterceptingClientHttpRequest.java:93)
org.springframework.http.client.InterceptingClientHttpRequest.executeInternal(InterceptingClientHttpRequest.java:77)
org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:739)
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:674)
org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:612)
com.TestRestService.getData(TestRestService.java:58)
com.TestRestService$$FastClassBySpringCGLIB$$91579cd4.invoke(<generated>)
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:156)
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749)
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691)
com.TestRestService$$EnhancerBySpringCGLIB$$f08eacac.getData(<generated>)
com.TestService.getData(TestService.java:25)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
java.base/java.lang.Thread.run(Thread.java:834)

Below the definition of resttemplate in Service A.

public RestTemplate restTemplate(final RestTemplateBuilder  builder) {
        return builder
                .requestFactory(new ClientHttpRequestFactorySupplier())
                .setConnectTimeout(Duration.ofMinutes(15))
                .setReadTimeout(Duration.ofMinutes(15))
                .build();
    }
skc
  • 65
  • 7
  • This is happening how long after the call? Is it always same? – mystery Nov 20 '21 at 03:47
  • Service B is taking less than 2 mins to respond the request. Added definition of resttemplate to the question. – skc Nov 20 '21 at 05:38
  • Is there any error in Service B? Usually this happens when server(Service B) close the connection – mystery Nov 20 '21 at 06:09
  • Yes, its 'java.io.IOException: Broken pipe' – skc Nov 20 '21 at 07:20
  • As I get, RestTemplate no disagned for processing big files. Probably you should consider usage of alternative clients which support streaming - https://stackoverflow.com/a/54484348/8312604 – Alexandr Arhipov Nov 20 '21 at 11:15
  • We got the root cause of this issue. We are deploying our applications to cloud environment and it has default route timeout as 30 seconds. As request was taking more than 30 seconds, so pod was closing the connection. After increasing the timeout, the issue got resolved. Thank you all for your help/answers. – skc Nov 22 '21 at 20:14

0 Answers0