1

I am new to Spring and Hibernate world. The issue is killing us. It's happening only in Production. We tried to reproduce this in a lower environment with the same load, configurations. But, it's not reproduced.

Back Round of the issue - When we are loading a set of Data from DB. Hibernate is flushing the session before reading. During that flush, we are facing this issue. We understood that we can handle it with Flush Mode or Read-only transaction or mutable entity (if read-only tables). But really, problem, since it's not reproduced in lower Env, we are not able to mark any of the above as a fix.

Any suggestion, help is much appreciated.

[5/1/18 14:33:00:774 EDT] 0000008b ThreadMonitor W WSVR0605W: Thread "WebContainer : 23" (00000142) has been active for 758291 milliseconds and may be hung. There is/are 19 thread(s) in total in the server that may be hung. at java.util.IdentityHashMap.put(IdentityHashMap.java:440) at org.hibernate.engine.internal.StatefulPersistenceContext.addChildParent(StatefulPersistenceContext.java:1657) at org.hibernate.engine.internal.Cascade.cascadeToOne(Cascade.java:456) at org.hibernate.engine.internal.Cascade.cascadeAssociation(Cascade.java:383) at org.hibernate.engine.internal.Cascade.cascadeProperty(Cascade.java:193) at org.hibernate.engine.internal.Cascade.cascade(Cascade.java:126) at org.hibernate.event.internal.AbstractFlushingEventListener.cascadeOnFlush(AbstractFlushingEventListener.java:150) at org.hibernate.event.internal.AbstractFlushingEventListener.prepareEntityFlushes(AbstractFlushingEventListener.java:141) at org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:74) at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:44) at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1396) at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1481) at org.hibernate.query.internal.AbstractProducedQuery.doList(AbstractProducedQuery.java:1445) at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1414) at org.hibernate.query.Query.getResultList(Query.java:146) at com.bbh.asl.service.dao.AslSearchDaoImpl.findAllAslFormByStatus(AslSearchDaoImpl.java:198) at com.bbh.asl.service.search.ASLSearchServiceImpl.findByAslStatus(ASLSearchServiceImpl.java:142) at com.bbh.asl.service.search.ASLSearchServiceImpl.getAslDashBoardData(ASLSearchServiceImpl.java:190) at sun.reflect.GeneratedMethodAccessor481.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213) at com.sun.proxy.$Proxy148.getAslDashBoardData(Unknown Source) at com.bbh.asl.web.service.ASLSearchServiceImpl.searchDashBoardData(ASLSearchServiceImpl.java:56) at com.bbh.asl.web.controller.ASLSearchController.loadDashBoard(ASLSearchController.java:69) at sun.reflect.GeneratedMethodAccessor480.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55) at java.lang.reflect.Method.invoke(Method.java:508) at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205) at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)

cosmoonot
  • 2,161
  • 3
  • 32
  • 38
Testuser
  • 11
  • 1
  • When this happens, please run "jstack -l pid" where pid is the process id of the Java process. In the stack traces you can find the thread that holds the lock (if this is a lock issue) and the threads that are waiting for it. What Java runtime are you using? – ewramner May 01 '18 at 19:02
  • Thank you for you response. Unfortunately. I dont have access to run it in production. Can you suggest more, if any. Suspicious part in this – Testuser May 01 '18 at 20:10
  • I guess you are on WAS? Are you sure it is configured in the same way in test (diff the xml for the config)? Can you set com.ibm.websphere.threadmonitor.dump.java=true in WAS in prod? That will produce a stack trace when the error occurs, giving you what jstack would produce, but it may be easier to convince operations to use the built-in function! Plus it could be a Java bug, in particular if you are using IBM's JDK. Make sure you have the latest fix packs from IBM. – ewramner May 02 '18 at 05:57
  • Yes, We are using WAS 9.0.0.6 and as per server team, They are keep saying everything same. I got the Thread dump and there are so many parked threads, but i could see any blocked one. And those threads does not seem to be related to this issue. Mostly from Spring, EhCache. Yes, we are using IBM JDK. IS there any known issue in 8.0.64 for identityhashmap – Testuser May 02 '18 at 22:29
  • Did you find any solution to this issue? We are facing a similar problem on JBoss 6.1 EAP on our PROD. – Kamil Roman May 22 '19 at 08:10

0 Answers0