JAVA Thread Dump: Too many waiting threads

Question

I'm facing difficulties in analyzing the thread dump of my vaadin 7.0 JAVA application and an integration layer written in spring MVC. There are too many threads in waiting state which is causing applications to slow down during peak hours and causing delays of upto 10 seconds in execution of simple code. Following is the trace which is appearing in waiting threads:-

priority:5 - threadId:0x00007f98b48de800 - nativeId:0x6511 - nativeId (decimal):25873 - state:WAITING
stackTrace:
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000006d5444af0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

I'm using API polling on many places in my application, following is the code of how I'm doing API polling:-

@Override
        public void run()
        {
            int counter = 1;

            while (true)
            {
                try
                {
                    if (counter == 1)
                        Thread.sleep(5000);
                    else
                        Thread.sleep(10000);

                    System.out.println("Call Some API");
                    if (counter == 3)
                        break;

                    counter++;
                }
                catch (Throwable e)
                {
                    new CustomException(e);
                }
            }

The applications are running on JAVA 1.8.0_65 and tomcat 8.0.5 Can anyone please guide me how can I fix this issue of too many waiting threads or how should I perform analysis for the resolution. For blocked threads I have already figured out the problem and fixed it.

That's a executor worker thread waiting for work to do, it's supposed to be in waiting state. Your thread count does seem high, but it's impossible to determine from that small piece of code what you're doing wrong. Leaking threads or threadpools possibly. — Kayaman, Nov 28 '19 at 08:25
Apart for using `Thread.sleep` is a bad idea, how do you actually start this polling? — daniu, Nov 28 '19 at 08:33
This is how I start my thread `ExecutorService executor = Executors.newFixedThreadPool(1); Runnable worker = new MyClassThread(); executor.execute(worker);` This same code is repeating in multiple JAVA files for API polling purposes. — Bilal Sheikh, Nov 28 '19 at 09:06

Kalz · Answer 1 · 2020-12-22T05:49:34.530

0

With thread.sleep() threads are still alive however in suspended mode. Under heavy load, you must be hitting your maximum thread limit as the system would need to create new threads as existing threads haven't completed their current work due to sleeping. This is the cause of high response time.

Increasing the number of threads should fix this but beware of your systems hardware limits. If you reaching your CPU/Mem limits then you'll have to add more servers to increase your throughput. I'm pretty sure that if reduce the sleep time, your app's performance will improve but then that may be your requirement in the first place.

Seems like more waiting threads is a side effect of more threads being active as CPU has to switch between them.

edited Dec 22 '20 at 05:49

answered Dec 22 '20 at 05:44

Kalz

1
2

but increasing the thread limit would be like putting duct tape on the hole. I think this code needs a lot of attention. – Tharaka Devinda Dec 22 '20 at 06:06
@Tharaka yes, we optimized the code. There were many resources that were not being used and discarded properly. We fixed them all and we were able to reduce these waiting threads. – Bilal Sheikh Mar 21 '22 at 14:03

JAVA Thread Dump: Too many waiting threads

1 Answers1