I am using GridDB as my in-memory database, and I intend to run heavy SQL queries on it. The underlying data has many columns/properties, distributed in many tables. Think of it as a large number of CSVs, with each CSV having 20 or more columns.
My SQL queries will join many tables to get the desired aggregtion results.
As per GridDb documentation there are 3 variables that can be set to improve quesry performance. storeMemoryLimit - Upper memory limit for intermediate data held in memory by SQL processing. workMemoryLimit - Upper memory limit for operators in SQL processing Restart workCacheMemory - Upper size limit for cache without being released after use of work memory.
I am a bit confused between workMemoryLimit and workCacheMemory. Does workCacheMemory kick in after workMemoryLimit is exhausted ? OR storeMemoryLimit is for intermediate data, workMemoryLimit is to be consumed by SQL operators only , and workCacheMemory is for everything else that might need to be put in cache ( e.g. results of an intermediate/inner query) ?
Thanks
Fine tuning my in-memory database for faster querying , given multiple tables with many columns
