JBoss AS Web 7.0.2 Broken pipe exception disconnects application and db connections

Question

My Server Configuration is as:

• JBOSS AS WEB 7.0.2
• Java (version "1.6.0_34" OpenJDK Runtime Environment (IcedTea6 1.13.6) (rhel-1.13.6.1.el6_6-x86_64) OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)

Also, My application is on HTTPS.

What is happening is, I am getting the below error in my log file and My Application is running and there is no high memory or CPU consumption and my hard disk is sufficient but the connectivity of my Application with Database is not there (All connections are gone). So whenever I am opening a page that uses DB is not opening and the page loader is loading and loading.

**ClientAbortException:  java.net.SocketException: Broken pipe
  at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:403)
  at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:449)
  at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
  at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:426)
  at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:415)
  at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:89)
  at org.codehaus.jackson.impl.Utf8Generator._flushBuffer(Utf8Generator.java:1754)
  at org.codehaus.jackson.impl.Utf8Generator._writeBytes(Utf8Generator.java:1164)
  at org.codehaus.jackson.impl.Utf8Generator._writeFieldName(Utf8Generator.java:457)
  at org.codehaus.jackson.impl.Utf8Generator.writeFieldName(Utf8Generator.java:304)
  at org.codehaus.jackson.map.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:444)
  at org.codehaus.jackson.map.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:150)
  at org.codehaus.jackson.map.ser.BeanSerializer.serialize(BeanSerializer.java:112)
  at org.codehaus.jackson.map.ser.std.StdContainerSerializers$IndexedListSerializer.serializeContents(StdContainerSerializers.java:122)
  at org.codehaus.jackson.map.ser.std.StdContainerSerializers$IndexedListSerializer.serializeContents(StdContainerSerializers.java:71)
  at org.codehaus.jackson.map.ser.std.AsArraySerializerBase.serialize(AsArraySerializerBase.java:86)
  at org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:610)
  at org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256)
  at org.codehaus.jackson.map.ObjectMapper.writeValue(ObjectMapper.java:1613)
  at org.springframework.http.converter.json.MappingJacksonHttpMessageConverter.writeInternal(MappingJacksonHttpMessageConverter.java:153)
  at org.springframework.http.converter.AbstractHttpMessageConverter.write(AbstractHttpMessageConverter.java:181)
  at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.writeWithMessageConverters(AnnotationMethodHandlerAdapter.java:975)
  at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.handleResponseBody(AnnotationMethodHandlerAdapter.java:933)
  at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter$ServletHandlerMethodInvoker.getModelAndView(AnnotationMethodHandlerAdapter.java:882)
  at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:428)
  at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:414)
  at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
  at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
  at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
  at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:734)
  at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
  at org.displaytag.filter.ResponseOverrideFilter.doFilter(ResponseOverrideFilter.java:125)
  at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
  at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:311)
  at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:116)
  at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:83)
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
  at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
  at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:101)
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
  at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
  at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
  at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
  at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
 at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:323)
 at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:173)
 at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:237)
 at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:167)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:280)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
 at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:139)
 at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:480)
 at org.jboss.as.web.NamingValve.invoke(NamingValve.java:57)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:154)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:362)
 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:877)
 at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:667)
 at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:952)
 at java.lang.Thread.run(Thread.java:701)
Caused by: java.net.SocketException: Broken pipe
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
 at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
 at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:375)
 at sun.security.ssl.OutputRecord.write(OutputRecord.java:363)
 at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:813)
 at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:785)
 at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:119)
 at org.apache.coyote.http11.InternalOutputBuffer.realWriteBytes(InternalOutputBuffer.java:726)
 at org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:449)
 at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:349)
 at org.apache.coyote.http11.InternalOutputBuffer$OutputStreamOutputBuffer.doWrite(InternalOutputBuffer.java:750)
 at org.apache.coyote.http11.filters.ChunkedOutputFilter.doWrite(ChunkedOutputFilter.java:126)
 at org.apache.coyote.http11.InternalOutputBuffer.doWrite(InternalOutputBuffer.java:559)
 at org.apache.coyote.Response.doWrite(Response.java:594)
 at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:398)
 ... 76 more**

I have checked my code and thought un-closed OutputStream was causing this issue, so I properly closed all. But still this issue persists.

Can anyone help me understand and provide the solution of this issue ?

Below is my current DataSource:

<bean id="myDataSource" class="com.jolbox.bonecp.BoneCPDataSource" destroy-method="close"> <property name="driverClass" value="com.mysql.jdbc.Driver" /> <property name="jdbcUrl" value="jdbc:mysql://${host}:${port} /${db_name}?autoReconnect=true" /> <property name="username" value="${user}" /> <property name="password" value="${password}" /> <property name="idleConnectionTestPeriodInSeconds" value="10" /> <property name="idleMaxAge" value="240" /> <property name="maxConnectionsPerPartition" value="30" /> <property name="minConnectionsPerPartition" value="10" /> <property name="partitionCount" value="3" /> <property name="acquireIncrement" value="5" /> <property name="acquireRetryAttempts" value="30" /> <property name="statementsCacheSize" value="100" /> <property name="releaseHelperThreads" value="3" /> <property name="connectionTestStatement" value="/* ping *\/ SELECT 1" /> <property name="acquireRetryDelayInMs" value="10000" /> </bean>

Answer 1

This is caused by writing to a connection when the other end has already closed it. DB connection is closed by database but for your application it is open so when it tries to write on it, it gives broken pipe error

Edit Adding my datasource details which not gives this error :

<Resource name="jdbc/MySQLDS"
              url="jdbc:mysql://${MYSQL_DB_HOST}:${MYSQL_DB_PORT}/${APP_NAME}"
              driverClassName="com.mysql.jdbc.Driver"
              username="${MYSQL_DB_USERNAME}"
              password="${MYSQL_DB_PASSWORD}"
              auth="Container"
              type="javax.sql.DataSource"
              initialSize="11"
              maxActive="20"
              maxIdle="15"
              minIdle="11"
              maxWait="10000"
              removeAbandoned="true" 
              removeAbandonedTimeout="60"
              testOnBorrow="true"
              validationQuery="SELECT 1"
              validationInterval="3600000"
              />