1

I am doing load testing on my application using jmeter and I have a situation where the cpu usage by the applications jvm goes to 99% and it stays there. Application still work, I am able to login and do some activity. But, it’s understandably slower.

Details of environment:

Server: AMD Optrom, 2.20 Ghz, 8 Core, 64bit, 24 GB RAM. Windows Server 2008 R2 Standard

Application server: jboss-4.0.4.GA

JAVA: jdk1.6.0_25, Java HotSpot(TM) 64-Bit Server VM

JVM settings: -Xms1G -Xmx10G -XX:MaxNewSize=3G -XX:MaxPermSize=12G -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+UseCompressedOops -Dsun.rmi.dgc.client.gcInterval=1800000 -Dsun.rmi.dgc.server.gcInterval=1800000

Database: MySql 5.6 (in a different machine)

Jmeter: 2.13

My scenario is that, I make 20 users of my application to log into it and perform normal activity that should not be bringing huge load. Some, minutes into the process, JVM of Jboss goes up and it never comes back. CPU usage will remain like that till JVM is killed.

To help better understand, here are few screen shots.

Visual GC

visual VM

I found few post which had cup @ 100%, but nothing there was same as my situation and could not find a solution. Any suggestion on what’s to be done will be great.

Regards,

Sreekanth.

Sreekanth
  • 380
  • 3
  • 14
  • You do not give any information of the kind of application and technologies you run on jboss e.g. libraries used or if it does some intense calculation... For proper testing results I would suggest you **not run visualvm and jmeter** on the same machine running the jboss application server. Run them on different servers so the impact of these two programs on your tomcat instance is reduced. – Tobi Tiggers May 12 '15 at 12:22
  • 1
    Looks like your application threads remain active after the user interactions end. Maybe you should look at reducing session keep-alive type settings? Are there any async operations your users start that keep running in the background? – RaGe May 12 '15 at 14:47
  • ThankYou, @dakillerbee. Our application is pretty huge and it does lot of conditional checking’s or calculations. Lot of lib files are also included. But, I may not be able to disclose more info. I thought visualvm and jmeter won’t cause much of problem as they are separate vm's. Will try and observe what you suggested. – Sreekanth May 13 '15 at 07:27
  • Thanks, @RaGe. There are few threads that are active throughout and I believe they are scheduled. Had run a analysis of them from jxm when I saw similar replies in other posts of cpu @ 100%. Do you have any better suggestion on how to identify a possible trouble making thread? – Sreekanth May 13 '15 at 07:29
  • Your JBoss console should give you a bit more information about what threads are active and for how long etc - possibly even how much CPU time each thread is taking. – RaGe May 13 '15 at 13:57

1 Answers1

0

To understand the root cause of the high CPU utilization, we need to check the CPU data and thread dumps at same time.

Capture 5-6 thread dumps at the time of the issue. Similarly capture CPU consumption thread-by-thread basis.

Generally the root cause of the CPU issue would be problems with threads like BLOCKED threads, long running threads, dead-lock, long running loops etc. That can be resolved by going through the stacks of the threads.

Swapnil Chaudhari
  • 31
  • 1