Springboot increasing threads even if server.tomcat.max-threads=5

Question

I have a springboot application right now that the threads keeps on increasing even if the server is idle. This causes the machine to crash over time. Causing very high memory and CPU usage.

I tried setting server.tomcat.max-threads=5 but I am very unlucky and the threads keeps on rising.

By the way, these are my components: - Rest Template - OK HTTP - Hikari CP - Springboot and Spring Framework (@RestController and @Service) - HttpInterceptor (Request and Outgoing) - Hibernate / JPA

I am also counting the threads using htop and the JNI I have created. The JNI PID/TID can get htop's light-weight-process IDs. So this JNI can virtually map that LWPID and the java thread.

Any help will be much appreciated.

Thanks,
Artanis Zeratul

How are you counting your threads? How many do you expect? How many are you observing? — Christopher Schultz, Dec 07 '18 at 14:40
Are you sure that it’s Tomcat creating the threads? Perhaps you can share some more information about their names and stack traces. — Andy Wilkinson, Dec 07 '18 at 22:58
@ChristopherSchultz, thanks for the feedback I appreciate it. By the way, I have edited my question. If you need more info I will just inform you or I can just edit my question. Thanks. — Artanis Zeratul, Dec 08 '18 at 08:59
@AndyWilkinson, thanks for the feedback too. That I am not so sure also probably some other components or 3 party components. Also but I don't have any means right now. I am working on Visual VM to generate thread dumps. — Artanis Zeratul, Dec 08 '18 at 23:13
If you take a thread-dump, what are the names of the threads that are running (and accumulating)? — Christopher Schultz, Dec 09 '18 at 15:51

score 0 · Answer 1 · answered Dec 12 '18 at 03:25

So I bet tomcat only has 5 threads, and some other technology you have in your server is creating those additional threads. Now tomcat follows a naming scheme for its threads. Typically those 5 threads would be named something like:

http-bio-8080-exec-1
http-bio-8080-exec-2
http-bio-8080-exec-3
...

You get the picture. So if you list the names of the threads in your VM (ctrl-page-break on windows, kill -3 JAVA_PID on linux, jconsole - probably your best bet. You should only see that thread naming pattern for 1-5. All other threads would be owned by something else. Tomcat does have extra threads beyond the exec threads, but they are fixed number (stop thread, main, etc).

score 0 · Accepted Answer · answered Dec 24 '18 at 02:52

My problem solved now.
These were the things I have observed and were the root causes of the problem.

Firstly, I have created Hikari Thread Pool and Local Entity Manager Factory Bean for every query I made which should only be done once in the application's lifetime or once for every datasource connection. This was the major cause of threads keeps spawning up. Creation of Thread Pool/LocalEntityManagerFactoryBean pertains to this code snippet:

HikariDataSource connectionPoolDatasource = new HikariDataSource(connectionPoolConfig);

localContainerEntityManagerFactoryBean.setDataSource(connectionPoolDatasource);
localContainerEntityManagerFactoryBean.setJpaProperties(jpaProperties);
localContainerEntityManagerFactoryBean.afterPropertiesSet();

Reference: https://groups.google.com/forum/#!topic/hikari-cp/IsVEn-D6oFA
See Brett Wooldridge response to Jen-Ming Chung.

Lastly, I didn't call Connection.close() everytime I have made query or connection to the DB.
Which also leads to threads not being terminated or closed.
As well as leaving DB connections open.
Reference: Getting Database connection in pure JPA setup

Cheers!

Springboot increasing threads even if server.tomcat.max-threads=5

2 Answers2

Linked