web application hangs in tomcat 6.0.21/7.0.11 without clue

Question

I just have a web application which hangs after few minutes. Basically, this is an application to build and preview forms, very customized, and after few minutes of intensive user interaction, the application just stops working. This means, a request is done and never returns a reply.

I am "a bit" lost as I don't find any clue about the hang in tomcat logs and application logs either. The application server is running as I am using lambda/psi-probe and can check any other and even the troublesome application with no problem (probe itself is another web application).

The application uses hibernate, which was working ok for long time, and, recently, to improve the performance (hibernate injects a lot of queries if you work exclusively with it) I have introduced native sql via java.sql standard api. I am careful not to mix both and they are only used in a jsp, first retrieving some (few) objects with hibernate and then using some logic with jdbc. Hibernate session is closed before using jdbc.

I have read something about database connection problems (I've checked several times and database server is running ok), deadlocks or runaway threads, checked using VisualVM in VisualVM.

So, anybody can give a clue about finding or trapping the hang? can give something a clue to use VisualVM to catch or trap the hypothetical runaway thread or deadlock? The latter would enlighten me as I just see waiting and running threads.

I am using tomcat 6.0.21 (I always tried 7.0.11 with same results) java 1.6 on mac osx and linux (development and preproduction machines)

Any ideas will be welcome for sure thanks

w i l l y

Rich · Answer 1 · 2011-03-17T08:15:05.027

Try running jconsole (if you aren't able to do this on the machine that is suffering the hang, you will need to enable remote JMX, restart the JVM and reproduce the hang), and then click the Threads tab and click on Detect Deadlock. That may provide some help.

Other things to use are, as you say, VisualVM, or JCarder.

Edit

After seeing your comment, another thing to try is the following:

Start JConsole and connect to your hung VM
Navigate to MBeans → com.sun.management → HotSpotDiagnostic → Operations
There will be a button called dumpHeap with two text boxes next to it. In the first text box, enter a unique name - I use something like my initials and the date - rt20110317. Keep the second textbox set to true. Hit the dumpHeap button.

This will write the heap to a large file. Find this file and load it into Eclipse MAT. This tool provides all sorts of useful bits and pieces for looking at the state of the VM. There are even a few wizards for finding memory leaks which may also find the reason as to why your VM is hanging if it's due to the same kind of symptoms.

sorry, it didnt contribute anything new, app hangs and i dont see anything remarkable — w i l l y, Mar 16 '11 at 17:09

score 1 · Answer 2 · edited May 30 '11 at 11:48

1

I have been in situation similar to this. The problem was that some of the database connections were never returned to the connection pool. After the limit was reached, the application will just freeze, no exceptions or errors. It might be the case with you as well. Turn on abandoned connection logging in your connection pool, you might want to check if there are any rogue connections.

edited May 30 '11 at 11:48

Marcelo Cantos

181,030
38
327
365

answered Mar 16 '11 at 16:36

Satadru Biswas

1,555
2
13
23

I too have a similar problem .. Is it a bug in tomcat6? – KronnorK May 14 '11 at 17:30
Did anything worked for you guys? I have same problem my application stops responding after 12-18 hrs – Amol Ghotankar Jul 28 '13 at 15:24
@AmolGhotankar What is the situation of your heap space when the application hangs. Try finding out any memory leak in your code. Also check for unreturned connections to the database connection pool. – Satadru Biswas Jul 31 '13 at 04:13
the connections are never returned back to the pool. How to solve ? I'm using hibernate with c3p0 – coding_idiot Oct 12 '13 at 17:26

score 0 · Answer 3 · answered Jul 13 '11 at 18:23

make sure you close any JDBC calls like this: Reference: http://tomcat.apache.org/tomcat-6.0-doc/jndi-datasource-examples-howto.html#Preventing_dB_connection_pool_leaks

Here is an example of properly written code to use a db connection obtained from a connection pool:

  Connection conn = null;
  Statement stmt = null;  // Or PreparedStatement if needed
  ResultSet rs = null;
  try {
    conn = ... get connection from connection pool ...
    stmt = conn.createStatement("select ...");
    rs = stmt.executeQuery();
    ... iterate through the result set ...
    rs.close();
    rs = null;
    stmt.close();
    stmt = null;
    conn.close(); // Return to connection pool
    conn = null;  // Make sure we don't close it twice
  } catch (SQLException e) {
    ... deal with errors ...
  } finally {
    // Always make sure result sets and statements are closed,
    // and the connection is returned to the pool
    if (rs != null) {
      try { rs.close(); } catch (SQLException e) { ; }
      rs = null;
    }
    if (stmt != null) {
      try { stmt.close(); } catch (SQLException e) { ; }
      stmt = null;
    }
    if (conn != null) {
      try { conn.close(); } catch (SQLException e) { ; }
      conn = null;
    }
  }

"

web application hangs in tomcat 6.0.21/7.0.11 without clue

3 Answers3