I'm new to JBOSS and I have a memory issue with a JVM (running under JBoss) and I wanted to see if you could help?
My JBOSS server running Teiid has been running out of memory when querying against a large table (130 million rows with 45 columns).
Here are some details:
Jboss EAP 6.4 with Teiid 8.7.1
It's running on a dedicated AWS Windows Server instance with 244GB ram, 32 processors
I have the JVM arguments as below:
-server -d64 -XX:+AggressiveOpts -XX:+UseFastAccessorMethods -XX:+PrintGCDetails -Xms200g -Xmx200g -XX:NewSize=8g -XX:MaxNewSize=8g -XX:SurvivorRatio=8 -XX:MaxDirectMemorySize=10g -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+UseLargePages
Teiid Buffer Manager settings:
allow-env-function" => false,
"async-thread-pool" => "teiid-async",
"authorization-validator-module" => undefined,
"buffer-service-encrypt-files" => false,
"buffer-service-inline-lobs" => true,
"buffer-service-max-buffer-space" => 500000L,
"buffer-service-max-file-size" => 150000L,
"buffer-service-max-open-files" => 64,
"buffer-service-max-processing-kb" => -1,
"buffer-service-max-reserve-kb" => -1,
"buffer-service-max-storage-object-size" => 16777216,
"buffer-service-memory-buffer-off-heap" => true,
"buffer-service-memory-buffer-space" => -1,
"buffer-service-processor-batch-size" => 30000,
"buffer-service-use-disk" => true,
"detect-change-events" => true,
"distributed-cache-jgroups-stack" => undefined,
"exception-on-max-source-rows" => true,
"lob-chunk-size-in-kb" => 100,
"max-active-plans" => 20,
"max-row-fetch-size" => 20480,
"max-source-rows-allowed" => -1,
"max-threads" => 64,
"policy-decider-module" => "org.jboss.teiid",
"preparedplan-cache-enable" => true,
"preparedplan-cache-infinispan-container" => "teiid-cache",
"preparedplan-cache-name" => "preparedplan",
"query-threshold-in-seconds" => 600,
"query-timeout" => 0,
"resultset-cache-enable" => true,
"resultset-cache-infinispan-container" => "teiid-cache",
"resultset-cache-max-staleness" => 60,
"resultset-cache-name" => "resultset",
"thread-count-for-source-concurrency" => 0,
"time-slice-in-milliseconds" => 2000,
"workmanager" => "default",
As you can see I've assigned the Heap size to 200gb but it's still maxing out the memory every time and the server just hangs up with "Full GC(Ergonmics)" error. Below is a sample log:
[GC (Allocation Failure) [PSYoungGen: 7549952K->838656K(7549952K)] 176791593K->176710678K(208876544K), 64.7327072 secs] [Times: user=1476.97 sys=9.75, real=64.73 secs]
GC (Allocation Failure) [PSYoungGen: 7549952K->838656K(7549952K)] 183421974K->183391750K(208876544K), 68.1115075 secs] [Times: user=1554.99 sys=9.14, real=68.12 secs]
[GC (Allocation Failure) [PSYoungGen: 7549952K->838656K(7549952K)] 190103046K->190019046K(208876544K), 70.2219754 secs] [Times: user=1603.11 sys=9.98, real=70.23 secs]
[GC (Allocation Failure) [PSYoungGen: 7549952K->838656K(7549952K)] 196730342K->196631886K(208876544K), 87.7628514 secs] [Times: user=2007.72 sys=8.92, real=87.76 secs]
[Full GC (Ergonomics) [PSYoungGen: 838656K->0K(7549952K)] [ParOldGen: 195793230K->193496415K(201326592K)] 196631886K->193496415K(208876544K), [Metaspace: 130959K->130959K(139264K)], 429.0076056 secs] [Times: user=1582.83 sys=5.59, real=429.00 secs]
[GC (Allocation Failure) [PSYoungGen: 6711296K->838656K(7549952K)] 200207711K->200111591K(208876544K), 49.8578163 secs] [Times: user=1141.05 sys=3.48, real=49.86 secs]
[Full GC (Ergonomics) [PSYoungGen: 838656K->0K(7549952K)] [ParOldGen: 199272935K->200062941K(201326592K)] 200111591K->200062941K(208876544K), [Metaspace: 130999K->130999K(139264K)], 383.1849674 secs] [Times: user=570.09 sys=7.91, real=383.17 secs]
Full GC (Ergonomics) [PSYoungGen: 6711296K->5262703K(7549952K)] [ParOldGen: 200062941K->201326365K(201326592K)] 206774237K->206589069K(208876544K), [Metaspace: 131005K->131005K(139264K)], 573.4518437 secs] [Times: user=608.89 sys=15.22, real=573.45 secs]
