Maximum table/query size limits for Postgres Foreign Data Wrapper (FDW)?

Asked Mar 21 '19 at 20:03

Active Mar 30 '19 at 00:14

Viewed 653 times

Are there limits to the PostgreSQL Foreign Data Wrapper extension? i.e. is there some maximum size of source table (or schema/database) that can be set up as a Foreign Table via FDW? We are using Postgres 10.6

We've occasionally run into situations where it seems to fail with tables that cross some size threshold (~8GB) but I'm unsure if this is something we can tweak, or perhaps a limitation on the source db end. (In this example the source table is in Redshift.)

The error message that comes back is: ERROR: exceeded the maximum size allowed for the total set of cursor data: 8000MB.

EDIT: Discovered the source of the problem - the size of this particular Redshift instance has this cursor size limit. They are listed here: https://docs.aws.amazon.com/redshift/latest/dg/declare.html#declare-constraints

We ended up filtering the results to make the FDW more workable.

edited Mar 30 '19 at 00:14

asked Mar 21 '19 at 20:03

rocksteady

1,697
14
18

3

Can you give example of the error message/sql used when the error occurs. I use somw large redshift FDWs and have not hit a problem yet – Jon Scott Mar 21 '19 at 20:23
How are you running PostgreSQL? On an EC2 instance? RDS? Aurora? – Nathan Griffiths Mar 22 '19 at 00:35
Error message added - hitting a max size for cursor data. And our Postgres instance is running on RDS. – rocksteady Mar 27 '19 at 13:25

Maximum table/query size limits for Postgres Foreign Data Wrapper (FDW)?

0 Answers0