16

Thread count in my tomcat application server is growing every day.

When I have taken thread dump for analysis.

I found that out of 430 threads 307 threads are with this status.

Sample stacktrace

"pool-283-thread-1" #2308674 prio=5 os_prio=0 tid=0x000000000a916800 nid=0x1101 waiting on condition [0x00002aec87f17000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000006d9929ec0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
    - None

"pool-282-thread-1" #2307106 prio=5 os_prio=0 tid=0x000000000a4fb000 nid=0x78e3 waiting on condition [0x00002aec87e16000]
   java.lang.Thread.State: WAITING (parking)
    at sun.misc.Unsafe.park(Native Method)
    - parking to wait for  <0x00000006d8ca7bf8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

   Locked ownable synchronizers:
    - None

Environment

JDK: jdk1.8.0_60
OS: Linux
Tomcat: tomcat-7.0.65

Not sure if this is causing the issue.

Appreciate any help on this.

Patan
  • 17,073
  • 36
  • 124
  • 198
  • 2
    Those threads appear to be a thread pool waiting for a task in a queue. I think you'll have to tell us how you setup that thread pool and queue. – WW. Oct 17 '16 at 06:47
  • 1
    Do you happen to miss a shutdown on a threadpool/ExecutorService? – Fildor Oct 17 '16 at 07:12

2 Answers2

22

This is typical resource leak. You are using some sort of ExecutorService somewhere in your application and you are not closing that pool after work is done causing threads to await forever.

You should call ExecutorService#shutdown() to close pool and release/terminate its threads after work is done.

Threads names like pool-282-thread-1 pool-283-thread-1 suggests that you are most probably using single thread pool executor (because pool no. is large and thread no. is only 1). The idea behind ExecutorService is to reuse threads that are idle to do some more work. So insteed of createing new ExecutorService each time you need to do some background work, you should rather share single instance and use it in your application.

Antoniossss
  • 31,590
  • 6
  • 57
  • 99
0

This issue happened in our application. We fixed it by using single ExecutorService instead of creating ExecutorService each time.

Aksen P
  • 4,564
  • 3
  • 14
  • 27
  • Hi Krishnan, welcome to SO. Please be sure to read the rules while you're here. Answers should be left to answers to the question, not +1 or thanks posts. If you could add some code to show your solution it would help this post a lot. Cheers! – Jake Luby Oct 24 '19 at 19:39