2

We're trying to profile a remote tomcat application running in production. The problem is that all the threads in the web pool are blocked and this seems to prevent us from connecting with jconsole, jmc and even YourKit. All these tools work fine when the jvm is running fine.

The error from jconsole is a timeout: 

Could not connect to server1:9090 : Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: 
java.net.SocketTimeoutException: Read timed out]
Could not connect to server1:9090. Make sure the JVM is running and that you are using the correct protocol in the Service URL (service:jmx:rmi:///jndi/rmi://server1:9090/jmxrmi).

This makes it hard to figure out what's wrong with our application. The server doesn't use much cpu and there is free memory. So no apparent lack of resources. The jvm just seems dead

Java version : jdk1.7.0_75 Tomcat version : 7.0.65

Any ideas on how to connect to the jvm when it is like this?

Tommy
  • 4,011
  • 9
  • 37
  • 59
  • 1
    Maybe you can try `jstack -F ` to force a thread dump. – Beck Yang Mar 28 '16 at 14:06
  • Anything in the logs? How about the working directory of the JVM process? What if you enable `-XX:+HeapDumpOnOutOfMemoryError`? It's possible that the JVM is just so braindead it can't pick-up the phone... – Christopher Schultz Mar 28 '16 at 21:39
  • start with jstack -F stack-trace.log – Sanj Mar 29 '16 at 03:07
  • @ChristopherSchultz: it's not a memory issue. We think it's something withe the classloader in hibernate during high trafic – Tommy Mar 29 '16 at 16:40

1 Answers1

0

You have a couple of options:

  1. Send SIGQUIT (Linux) or Control-Break (Windows) and get a stack trace.
  2. Get a core dump (gcore on Linux) and then you can use the JVM Tools like jstack and friends using the core file
sunnychan77
  • 91
  • 4