The story:
We have SOA Suite domain, 3 nodes. OEL Linux 6.8, CPU 32 cores. We have cleaned tmp folders, soainfta database tables truncated. Right after the domain start(SOA Platform is running...) we observe the domain degradation:
1) High CPU utilization (50-70%) on all nodes by jvm processes. Usual utilization was 10-15%. GC is G1, it is working OK, not taking CPU time.
2) In Enterprise manager of domain we observe that processes stuck on BPEL activation.
3) After 600 seconds we observe stuck threads in weblogic console, we observe transactions rollback in logs.
4) We observe growth of Runnable and Blocked JVM threads
5) If we stop the load , then after few hours stucked instances go further the BPEL processes and they end with transaction timeouts, JVM threads are released.
6) We observe no problems on Database, nor we have JMS queues growth nor memory problems.
Probably, someone had the same issues and knows how to solve them. Please point to the right direction. Thanks.