How to store RDS data in S3 in near real-time? (aws dms)

Question

My goal is to store RDS data in real time on S3.

The one I found was the aws dms service, but it is not documented exactly how it works, so I am hesitant to introduce it.

What I am considering is Full load + CDC. However, I don't know how the full load works. Is it simply select * from table?

I am wondering if there is any problem with the server currently running by doing full load.

For example, a deadlock occurs because of a table lock due to select.

I have no idea how to process this log to make it into the final table.

The final goal is to configure the CDC log so that it can be queried from AWS athena with transformed data.

RDS: MariaDB version 10.2.21

score 0 · Answer 1 · answered Jul 24 '20 at 10:31

0

I recommend using aws data pipeline as you can export data as csv and store it into s3

you can schedule the job according to your requirements:

answered Jul 24 '20 at 10:31

AWS PS

thank you for answer. but data pipeline not supported in my region – kang Jul 28 '20 at 00:50

1 Answers1