4

I need advice from experienced jetty users.

I maintain 2 linux machines (with Jetty 9.0.3) behind a load balancer (Amazon cloud). Once in a while my Jetty containers are shutting down for no reason by 'Thread-2'. Simultaneously.

The below log is displayed and the container stops without reason. No errors. No exceptions. Gracefully shuts down. -- This is already weird! But... both 2 jettys from 2 machines are getting down at the same time... ???

No reason shutdown log (node 1):

2013-09-24 18:51:19.447:INFO:oejs.ServerConnector:Thread-2: Stopped ServerConnector@22480241{HTTP/1.1}{0.0.0.0:2323} ...

2013-09-24 18:51:23.443:INFO:oejsl.ELContextCleaner:Thread-2: javax.el.BeanELResolver purged 2013-09-24 18:51:23.443:INFO:oejsh.ContextHandler:Thread-2: Stopped o.e.j.w.WebAppContext@5892d4a8{/,file:/home/ec2-user/jetty/webapps/ROOT/,UNAVAILABLE}{/ROOT}

No reason shutdown log (node 2):

2013-09-24 18:51:22.152:INFO:oejs.ServerConnector:Thread-2: Stopped ServerConnector@ba4bb9{HTTP/1.1}{0.0.0.0:2323} ...

2013-09-24 18:51:25.605:INFO:oejsl.ELContextCleaner:Thread-2: javax.el.BeanELResolver purged 2013-09-24 18:51:25.605:INFO:oejsh.ContextHandler:Thread-2: Stopped o.e.j.w.WebAppContext@460434{/,file:/home/ec2-user/jetty/webapps/ROOT/,UNAVAILABLE}{/ROOT}

Normal shutdown log for comparison (start.jar --stop)

2013-09-25 16:25:16.993:INFO:oejs.ServerConnector:ShutdownMonitor: Stopped ServerConnector@ba4bb9{HTTP/1.1}{0.0.0.0:2323} ...

2013-09-25 16:25:21.049:INFO:oejsl.ELContextCleaner:ShutdownMonitor: javax.el.BeanELResolver purged 2013-09-25 16:25:21.049:INFO:oejsh.ContextHandler:ShutdownMonitor: Stopped o.e.j.w.WebAppContext@460434{/,file:/home/ec2-user/jetty/webapps/ROOT/,UNAVAILABLE}{/ROOT}

Note that 'Thread-2' is stopping the server for no clear reason, and not the 'ShutdownMonitor' thread. What could be happening? Is there some hidden door that could stop the container other than running 'start.jar' with --stop?

Please help!

Update:

Well, I found out that if I kill the process instead of calling 'stop', the result is the same... thread-2. So I believe linux is killing my process. Perhaps OOM. I'm researching. Perhaps this question is no more needed. Thanks

Updade 2:

The error was that I was starting the process in my terminal (without &), so it was running in the foreground - bound to that terminal. When the terminal session ended, the process was terminated.

Revo

MarqueHora Agendamentos
  • 81
  • 1
  • 6
  • Not much to go on here. Include more context in your logs. Maybe a few lines above the ShutdownMonitor is the key piece of information you need. – Joakim Erdfelt Sep 25 '13 at 20:18
  • Those are the only logs that are printed. But please check my update, I guess the process is being killed. Thanks anyway! – MarqueHora Agendamentos Sep 27 '13 at 20:04
  • 1
    check out /var/log/messages on the host that is being killed. The AWS oom-killer is supposed to log there when it kills something. – Talandar Sep 27 '13 at 20:08
  • Now I encounter the same problem, no error logs of webapp, no OOM logs of jvm and system, just one line of log: `2015-06-18 20:17:02.354:INFO:oejs.ServerConnector:Thread-2: Stopped ServerConnector@18d9581a{HTT P/1.1}{0.0.0.0:8080}`, do you resolve the problem? please give me some tips, thank you! – Michael Jun 18 '15 at 12:56

0 Answers0