Get incremental changes for RDS Postgres DB

Asked Aug 16 '19 at 06:15

Active Aug 16 '19 at 06:20

Viewed 138 times

I have a process that extract data from origin DB, make changes and insert it to target DB.

Now I'm using an AWS Lambda that runs every few minutes and added a timestamp column that indicates when the data was last changed to filter base on it.

This process is inefficient since I need to "remember" to manually add a timestamp column to each new table. Is there a better way? can I use a query (Postgres) or an API call (boto3) to get only the data that was changed?

edited Aug 16 '19 at 06:20

asked Aug 16 '19 at 06:15

shlomiLan

Check https://stackoverflow.com/questions/6207603/database-mirroring-postgres-streaming-replication, might be of help if you all you're looking for is a mirroring setup. – Sanketh Aug 16 '19 at 06:27
Does logical replication work with RDS? – Laurenz Albe Aug 16 '19 at 06:42
This is what found https://cunyang.me/postgresql-cdc-aws-rds/ it sounds like a good way to investigate – shlomiLan Aug 16 '19 at 06:46
It has been a very long time but did you find any solution? – ORCAs Jul 03 '22 at 06:31
implemented a solution with kafak connect @ORCAs – shlomiLan Oct 23 '22 at 14:24

Get incremental changes for RDS Postgres DB

0 Answers0