5

I'm SO confusing about memory-settings in Presto. Please check this out below:

  1. query.max-memory

  2. query.max-memory-per-node (base config) enter image description here

  3. query.max-total-memory (release in 0.205) enter image description here

  4. resources.reserved-system-memory (admin properties) enter image description here

  5. Memory Pools (General Pool & Reserved Pool) enter image description here

That is all I can find out.

Here are my Presto settings:

  1. etc/config.properties

query.max-memory=2.25GB
query.max-total-memory=2.25GB
query.max-memory-per-node=0.75GB

  1. jvm: -Xmx3G -Xms3G

Base on my settings, I found these rules:

  1. query.max-memory-per-node <= jvm * 0.25

  2. General Pool = jvm * 0.4, same as the default value of resources.reserved-system-memory

  3. Reserved Pool = jvm * 0.3

Here are my questions:

  1. The relationship between all kinds of memory-settings. (like resources.reserved-system-memory is General Pool? query.max-total-memory=user memory+system memory, what are user memory and system memory?)

  2. What General Pool and Reserved Pool use for? How can I change their values?

  3. General Pool(40% of jvm), Reserved Pool(30% of jvm), where is the last 30% of jvm? or How Presto assigns my 3GB jvm memory?

Archon
  • 1,385
  • 1
  • 15
  • 30

1 Answers1

2
reserved = query.max-memory-per-node 
system = 40% of heap 
general = heap - system - reserved
Archon
  • 1,385
  • 1
  • 15
  • 30
  • 1
    From "about the memory manager" link above: The way it works is that the reserved pool gets X GB of memory, where X is the size of the largest query that could be submitted, and the general pool gets all the remaining memory. All submitted queries start in the general pool, but if it becomes full the largest is promoted to the reserved pool. Since the reserved pool has enough memory to run the largest query that query will always complete and therefore the cluster can never deadlock. – Damián Montenegro Oct 16 '20 at 21:17