0

I have an enterprise event-driven application. I have 16 MDBs that read from a JMS queue. When I got some uncontrolled exceptions I receive this stack :

####<Apr 18, 2011 11:47:15 AM CEST> <Error> <WebLogicServer> <sapmop11> <cont0Server11> <[ACTIVE] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1303120035886> <BEA-000337> <[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)' h
as been busy for "647" seconds working on the request "weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl@16b13f90", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
        java.lang.Thread.sleep(Native Method)
        weblogic.ejb.container.internal.JMSPollerManager.waitForPollersToStop(JMSPollerManager.java:82)
        weblogic.ejb.container.internal.JMSConnectionPoller.waitForMessagePollerExit(JMSConnectionPoller.java:795)
        weblogic.ejb.container.internal.JMSConnectionPoller.disconnect(JMSConnectionPoller.java:920)
        weblogic.ejb.container.internal.MDConnectionManager.timerExpired(MDConnectionManager.java:166)
        weblogic.timers.internal.TimerImpl.run(TimerImpl.java:273)
        weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
        weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
        weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
>
####<Apr 18, 2011 11:48:15 AM CEST> <Error> <WebLogicServer> <sapmop11> <cont0Server11> <[ACTIVE] ExecuteThread: '9' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1303120095935> <BEA-000337> <[STUCK] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)' h
as been busy for "707" seconds working on the request "weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl@16b13f90", which is more than the configured time (StuckThreadMaxTime) of "600" seconds. Stack trace:
        java.lang.Thread.sleep(Native Method)
        weblogic.ejb.container.internal.JMSPollerManager.waitForPollersToStop(JMSPollerManager.java:82)
        weblogic.ejb.container.internal.JMSConnectionPoller.waitForMessagePollerExit(JMSConnectionPoller.java:795)
        weblogic.ejb.container.internal.JMSConnectionPoller.disconnect(JMSConnectionPoller.java:920)
        weblogic.ejb.container.internal.MDConnectionManager.timerExpired(MDConnectionManager.java:166)
        weblogic.timers.internal.TimerImpl.run(TimerImpl.java:273)
        weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
        weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
        weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
>

After this all the MDBs stop reading from the queue and the application need to be restarted.

This is the MDB onMessage code:

@Override
    @TransactionAttribute(TransactionAttributeType.REQUIRED)
    public void onMessage(Message message) {
        OnlineEventMessage<? extends Serializable> eventMessage = null;
        try {
            TextMessage textMessage = (TextMessage) message;
            String stringMessage = textMessage.getText();

            logger.debug("e' stato ricevuto il messaggio {}", stringMessage);
            logger.debug("eventMessageParser {}", onlineEventMessageParser);
            logger.debug("legacyEventMessageService {}", legacyEventMessageService);

            EventMessageAnagrafica anagrafica = anagraficaParser.parse(stringMessage, true);
            String multichannelId = anagrafica.getMultichannelId();

            eventMessage = onlineEventMessageParser.parse(stringMessage, true);

            MDC.put("multichannelId", multichannelId);
            MDC.put("eventType", eventMessage.getEventType().getEventCode());
            legacyEventMessageService.handle(eventMessage);
        } catch (JMSException e) {
            retryJmsTemplate.send(destination, new RedeliveryMessageCreatorUtil(message));
            logger.error("si e' verificato un errore JMS nel processamento dell'evento {}, riaccodamento in corso...", e);
        } catch (RuntimeException e) {
            logger.error("si e' verificata una runtime exception per il messaggio {}", e);

            legacyErrorHandlerService.handle(eventMessage);

        } catch (Exception e) {
            logger.error("si e' verificato un errore inaspettato per il messaggio {}", e);
            if (eventMessage != null && eventMessage.getIdEventMessage() != null) {
                logger.error("Il messaggio gia' presente nelle catene di elaborazione");
                OnlineEventMessage<?> onlineMessage = eventMessageDao.getItemByID(eventMessage.getIdEventMessage());
                if (onlineMessage != null) {
                    onlineMessage.setEventState(EventState.PROCESSING_ERROR);
                    eventMessageDao.updateItem(onlineMessage);
                    eventMessageDao.flush();
                }
            }
            context.setRollbackOnly();
        }
    }

Any idea? Kind regards Massimo

MaDa
  • 10,511
  • 9
  • 46
  • 84
Massimo Ugues
  • 4,373
  • 8
  • 43
  • 56

1 Answers1

0

This seems like a transaction issue. Try taking a Thread Dump next time this happens and take a look at what the threads are doing. You can do this through the Admin Console.

Jeff West
  • 1,563
  • 9
  • 11