Attempting to connect Clickhouse to replicate data from PostgreSQL using https://clickhouse.com/docs/en/engines/database-engines/materialized-postgresql/. Any ideas on how to solve the error or what's the best way to replicate PostgreSQL data to Clickhouse?
CREATE DATABASE pg_db
ENGINE = MaterializedPostgreSQL('localhost:5432', 'dbname', 'dbuser', 'dbpass')
SETTINGS materialized_postgresql_schema = 'dbschema'
Then running SHOW TABLES FROM pg_db; doesn't show all tables (missing large tables that has 800k rows). When attempting to attach that large table using ATTACH TABLE pg_db.lgtable;, gets an error below:
Code: 619. DB::Exception: Failed to add table
lgtableto replication. Info: Code: 241. DB::Exception: Memory limit (total) exceeded: would use 1.75 GiB (attempt to allocate chunk of 4219172 bytes), maximum: 1.75 GiB. (MEMORY_LIMIT_EXCEEDED) (version 22.1.3.7 (official build)). (POSTGRESQL_REPLICATION_INTERNAL_ERROR) (version 22.1.3.7 (official build))
I've tried increasing allocated memory and adjusting other settings, but still getting the same problem.
set max_memory_usage = 8000000000;
set max_memory_usage_for_user = 8000000000;
set max_bytes_before_external_group_by = 1000000000;
set max_bytes_before_external_sort = 1000000000;
set max_block_size=512, max_threads=1, max_rows_to_read=512;
