Stack Overflow
Stack Overflow

Questions tagged [aws-dms]

AWS Database Migration Service (AWS DMS) can migrate your data to and from most widely used commercial and open-source databases such as Oracle, PostgreSQL, Microsoft SQL Server, Amazon Redshift, Amazon Aurora, MariaDB, and MySQL. The service supports homogeneous migrations such as Oracle to Oracle, and also heterogeneous migrations between different database platforms, such as Oracle to MySQL. The source or target database must be on an AWS service.

http://docs.aws.amazon.com/dms/latest/userguide/Welcome.html

465 questions
0
votes
0 answers

How to automation AWS Lambda/Data migration testing using selenium

I need to automate AWS DMS services using selenium, can someone please suggest is it possible to automate Amazon DMS testing using selenium. I am required to test source data with target data in AWS DMS.
satinder
  • 3
  • 2
0
votes
0 answers

MySQL Timestamps transferred using AWS-DMS seem to be a mixture of GMT and BST

I'm using AWS-DMS to copy a local MySQL database into an AWS-RDS MySQL database. I am finding the fields that are TIMESTAMP will differ from source to destination, but only for date/times that are in BST (British Summer Time - Daylight Saving…
Richard B
  • 1
  • 1
0
votes
1 answer

AWS DMS service: creating target endpoint for S3 is throwing error Role is not configured properly.AccessDenied

I am creating configuring a AWS DMS to migrate my RDS database to the S3 bucket. I created the instance and the source endpoint. There was no issue creating them. Now, I am creating a target endpoint choosing the S3 as the source. Before I create…
Wai Yan Hein
  • 13,651
  • 35
  • 180
  • 372
0
votes
1 answer

AWS DMS unable to write to MSK target

Note: self-answered question, because Google didn't shed any light on the problem. I have configured a Managed Streaming for Kafka target for AWS Data Migration Service, but the migration job fails. Looking at the logs, I see…
kdgregory
  • 38,754
  • 10
  • 77
  • 102
0
votes
1 answer

Can I configure AWS RDS to only stream INSERT operations to AWS DMS?

My requirement is to stream only INSERTs on a specific table in my db to a Kinesis data stream. I have configured this pipeline in my AWS environment: RDS Postgres 13 -> DMS (Database Migration Service) -> KDS (Kinesis Data Stream) This setup works…
raxaghi
  • 3
  • 2
0
votes
1 answer

AWS DMS Replication Instances on different Availability Zones Connection Issue

I am facing an issue with AWS DMS Replication Instance - it is not working on a specific AZ. Below explains the scenario and setup: AWS Account #1 MySQL RDS (Multi-AZ) AWS Account #2 DMS Replication Instance (on Availability Zone - A) DMS…
93Ramadan
  • 343
  • 3
  • 8
0
votes
1 answer

Error in Connecting kinesis as the target using dms

I am trying pass data aws s3-DMS-kinesis. I have uploaded a cab file in s3 bucket. I have successfully retrieved the data from s3 using DMS but when I am trying to connect the DMS target to kinesis, I am getting error “ failed to connect to…
sushma
  • 1
  • 1
0
votes
0 answers

How AWS DMS full load migration affect source database performance?

Do you know how an AWS DMS full load from a MySQL Aurora as a source database affects its performances? My cluster MySQL Aurora is used in production and cannot have any downtime or bad performance. I ve difficulties finding any clue on AWS…
felicienb
  • 83
  • 10
0
votes
2 answers

AWS DMS - column adding order

1)when I add a column can i specify its order in the table? A column was adding to the end of table, without this parameter 2) How i can mask data { "rule-type": "transformation", "rule-id": "2", "rule-name": "2", "rule-target":…
ti_key
  • 51
  • 1
  • 1
  • 6
0
votes
1 answer

AWS DMS - check volume of transfered data within time range

Is there a quick way to check how many data (volume wise, GBs, TBs etc) did my specific DMS task transfered for example within last month? I can't find any note in the documentation on that, I could probably try with boto3 but want to double check…
marcin2x4
  • 1,321
  • 2
  • 18
  • 44
0
votes
1 answer

AWS DMS fails to replicate data

Problem im facing - only some columns gets detected Created a source and target endpoints and a replication instance ( connection tested for both ) Want to copy a table called users that look like this : Mapping rules : I insert data on source…
Eyal Solomon
  • 470
  • 1
  • 6
  • 15
0
votes
1 answer

AWS DMS - Source S3, Target Aurora MySQL- Error- invalid utf8mb4 character string

My source is CSV files in S3 bucket and destination is Aurora MySQL 5.7. The database is defined with charset utf8 and the table also uses utf8. My source file is using latin1 (iso-8859-1) charset has special characters like ¢ and ° . I am getting…
Div
  • 11
  • 2
0
votes
1 answer

AWS DMS not able to migrate some tables between MySQL databases

I am migrating AWS RDS MySQL instance to another AWS RDS MySQL instance using Data migration service (DMS). Found below tables in mysql and performance_schema databases are not migrating and giving an error with pending validation in DMS table…
Prime
  • 3,530
  • 5
  • 28
  • 47
0
votes
1 answer

AWS DMS - DocumentDB > ElasticSearch - Error Getting Primary Key String

I am using DMS to periodically migrate all data from DocumentDB into a elasticSearch cluster. When I run the task, some tables migrate normally into ElasticSearch whereas in some I get errors. There are two errors that I am getting for every table…
Sumit Maingi
  • 2,173
  • 3
  • 24
  • 44
0
votes
1 answer

includeOpForFullLoad property on Parquet files - AWS DMS

I have tried includeOpForFullLoad property for csv files while reading from oracle and writing in to s3. I wanted to use this property for parquet files. I can see it works for Incremental but not on full load. Anyone tried ? ?
Srinivas Balaji
  • 21
  • 4
1 2 3
…
30 31