0

Linux: 16gb ram, 2cpus

On Linux machine running tomcat with around 800 threads which I got using jvisualVM, but in run queue length I am getting only 50 because CPU is 100%, and no blocked threads are showing. I got these value through NMON.

so following questions I have:

  1. According to my understanding for every java thread one kernel thread will be created, if it is so why I am not getting run queue length 800, as well No blocked threads.

2.So it means only 50 threads are waiting to run remaining threads are in running state, is it possible that 750 around threads are running at a time?

clear this confusion to me, I googled every where on this not got any clue also. references much appriciated

raghava
  • 1
  • From your question it's not clear what kind of workload you're trying. Are you doing a load test? If so, how much concurrent request per sec? – Fredi Apr 21 '16 at 10:35
  • Thanks for replay!. Yes I am doing load testing with JMeter with following is the load applied... for every 10 sec 10 vusers will be created, like that till 1500 users load is applied, from server side max threads is 800 and accept count is 2000, so it is accepting all the requests. so number of requests per second is around 2500. i want relation between threads it's showing in the jvisualvm to the run queue length. – raghava Apr 21 '16 at 10:37
  • @Fredi can you please tell from where I get solution to this problem – raghava Apr 21 '16 at 10:58
  • Another data point is needed, what is the mean response time from tomcat? You can enable the access log in server.xml with a setting that includes the duration, for ex: pattern='%h %u %t "%r" %s %b %I %D' – Fredi Apr 21 '16 at 11:13
  • And another important info, what kind of connector are you using? BIO / APR / NIO? – Fredi Apr 21 '16 at 11:14
  • @Fredi sorry, we conducted a test in last week, now we are analysing the results, we didn't enable the logs while testing, any chance to find out what is the reason? any references much appreciated!. – raghava Apr 21 '16 at 11:18
  • @Fredi the reason behind asking this question is since CPU is 100% utilized we need to increase the CPU's but how many CPU's? don't have any clear reference to tell. so once we find the relation between run queue length and threads that tomcat created, we can say how many CPU's we want right? am I thinking in a right direction? – raghava Apr 21 '16 at 11:26

1 Answers1

0

With those settings, assuming you're using the BIO connector (1 thread per connection), and a mean response time of 50 milliseconds (so each thread can give 20 replies per sec) you should be able to handle around:

(20 replies/sec * 800 threads) + 200 (accept count) + net.ipv4.tcp_max_syn_backlog (TCP stack) connections per second.

But as for your comment, you're interested in HW sizing. At this point, if you have the possibility to do other tests, as your CPU was 100% utilized, try retesting with half the load. Repeat increasing / decreasing the load till you reach 100% CPU with load average = number of cores. Now you have the maximum load that you can handle with the actual hardware. Knowing this and the expected load you now are able to know how much more CPU's you need.

Hope this helps.

Fredi
  • 2,257
  • 10
  • 13
  • When the load is around 10vusers the run queue length is 5 and CPU 60% when load hits to 50 then CPU is 100% utilized and run queue length is 50, from this point onwards the run queue length and CPU usage is constant till load the load increased to 1500 vusers. so to server 1500 users concurrently what CPU's u suggest? does the relation between run queue length with the threads help in this situation? – raghava Apr 21 '16 at 11:55
  • How much CPU's do you have for now? – Fredi Apr 21 '16 at 11:58
  • 2 CPU's, if you find how many CPU's required, please provide the how you calculated. – raghava Apr 21 '16 at 12:04
  • Are you measuring CPU usage with top? If so, as you have two CPU's it can go up to 200%. What sampling are you using? Seconds? – Fredi Apr 21 '16 at 14:49
  • There are plenty of variables as you can see, let's try though – Fredi Apr 21 '16 at 14:49
  • I am using nmon with 10sec, i don't have any idea what all those samples, so can you please suggest me. – raghava Apr 21 '16 at 16:10