0

I have recently installed a PostgreSQL 14.1 in parallel to my old 12.9 on my RedHat server. Both instances are running their default configurations. The server itself has 48 CPU and 188 GB RAM, which seemed to be more than sufficient for 12.9

Everything worked as expected, but I keep receiving an error message.

out of memory - Failed on request of size 24576 in memory context "TupleSort main"
SQL state: 53200

SQL tables: pos has 18 584 522 rows // orderedposteps has 18 rows // posteps has 18 rows

CREATE TEMP TABLE actualpos ON COMMIT DROP AS
SELECT  DISTINCT lsa.id 
FROM    pos sa
JOIN    orderedposteps osas ON osas.stepid = sa.stepid
JOIN    posteps sas ON sas.id = osas.stepid
JOIN LATERAL
    (
    SELECT  innersa.*
    FROM    pos innersa
    JOIN    orderedposteps innerosas ON innerosas.stepid = innersa.stepid
    WHERE   (innersa.id = sa.id) AND
        (innersa.iscached IS FALSE) AND
        (innersa.isobsolete IS FALSE)
    ORDER   BY innersa.createdtimestamp DESC, innerosas.stepindex DESC
    LIMIT   1
    ) lsa ON TRUE
LEFT JOIN LATERAL
    (
    SELECT  innersa.*
    FROM    pos innersa
    JOIN    orderedposteps innerosas ON innerosas.stepid = innersa.stepid
    WHERE   (innersa.id = sa.id) AND
        (innersa.iscached IS TRUE) AND
        (innersa.isobsolete IS FALSE)
    ORDER   BY innersa.createdtimestamp DESC, innerosas.stepindex DESC
    LIMIT   1
    ) sacheck ON TRUE
LEFT JOIN orderedposteps osascheck ON osascheck.stepid = sacheck.stepid 
WHERE   ((sacheck IS NULL) OR (sacheck.createdtimestamp < sa.createdtimestamp) OR (osascheck.stepindex < osas.stepindex))
AND     (((osas.stepindex < v_laststepindex) AND (sa.isfailure != sas.isvalidsum) AND (sa.iscached IS FALSE)) OR ((osas.stepindex = v_laststepindex) AND (sa.iscached IS FALSE)))
ORDER BY lsa.createdtimestamp DESC LIMIT 50000

The only difference I can see is the RAM utilization, showed by htop. While 12.9 only consumes up to 10 GB RAM, the 14.1 grows up to 62GB and crashes by reaching more or less 62GB.

I have already tried to increase the work_mem via

ALTER SYSTEM SET work_mem = '4MB';

Used pgtune as well in order to change some other values, but nothing has a significant effect.

I am pretty sure the SQL can be simplified and tuned, which I could do, but I want to understand where the difference between 12.9 and 14.1 is, or what to change configuration wise, instead of refactoring one function to work with the lasted version.

Laurenz Albe
  • 209,280
  • 17
  • 206
  • 263
user16479527
  • 1
  • 1
  • 2
  • `4MB` is the default size of `work_mem` which value did you set on the new server? –  Nov 29 '21 at 16:59
  • The log file should show a dump of the sizes of all memory contexts just after (or is it just before) that error. Where is all the space going? – jjanes Nov 29 '21 at 19:22
  • Please show an `EXPLAIN` plan for that query on both servers. For the one that doesn't crash, make it an `EXPLAIN (ANALYZE, BUFFERS)`. – jjanes Nov 29 '21 at 19:24
  • @a_horse_with_no_name we even set it to 1GB, but it doesn't seem to have any impact. – user16479527 Nov 30 '21 at 10:01
  • If you set it to 1GB it's not really a surprise that you get an OOM –  Nov 30 '21 at 10:02
  • @a_horse_with_no_name , we played a lot with work_mem between 0.5MB to 1GB, but there is always the same result – user16479527 Nov 30 '21 at 10:34

0 Answers0