Stack Overflow
Stack Overflow

Questions tagged [aws-dms]

AWS Database Migration Service (AWS DMS) can migrate your data to and from most widely used commercial and open-source databases such as Oracle, PostgreSQL, Microsoft SQL Server, Amazon Redshift, Amazon Aurora, MariaDB, and MySQL. The service supports homogeneous migrations such as Oracle to Oracle, and also heterogeneous migrations between different database platforms, such as Oracle to MySQL. The source or target database must be on an AWS service.

http://docs.aws.amazon.com/dms/latest/userguide/Welcome.html

465 questions
0
votes
0 answers

AWS DMS real time replication CDC from Oracle and PostgreSQL to Kinesis on a single thread is taking a lot of time

The goal: Real time CDC from Oracle and PostgreSQL to Kinesis on a single thread/process without much time lag and no record drop. The system: We have a system where we are doing a real time CDC from Oracle and PostgreSQL to Kinesis using AWS…
Curious_Insider
  • 27
  • 7
0
votes
1 answer

Triggering a Lambda once a DMS Replication Task has completed in Terraform

I would like to trigger a Lambda once an RDS Replication Task has successfully completed. I have the following Terraform code, which successfully creates all the assets, but my Lambda is not being triggered. resource "aws_dms_event_subscription"…
Rob
  • 145
  • 1
  • 13
0
votes
1 answer

Oracle UNUSABLE Index Problem With Bulk Insertion

I have a postgresql database and I have 1TB data. I want to migrate this data from postgresql to Oracle. I can handle it via DMS(Database Migration Service). When I migrate my data, Oracle indexes look like UNUSABLE then I try to rebuild it I got…
Sha
  • 921
  • 17
  • 46
0
votes
0 answers

How can AWS DMS perform ongoing replication without replication or CDC enabled on source SQL Server

According to https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html If your database isn't configured for MS-REPLICATION or MS-CDC, you can still capture tables that do not have a Primary Key, but only INSERT/DELETE DML events…
RMMD12
  • 33
  • 2
  • 8
0
votes
1 answer

Data mismatch in AWS DMS Bulk load vs CDC

I have a postgis database(source) for which i have done DMS and moved it to S3 bucket(target) in parquet file. There is a column name point of datatype-geometry(point, 4326) in the source which converts to string after dms and looks like this in…
Maab Khan
  • 1
0
votes
0 answers

Copying a source column to two destination columns in AWS DMS

I have a source database that I am trying to replicate using AWS DMS. In the source database there is table t with column c VARCHAR(100) I want to copy source.t.c into two columns in the destination: destination.t.c and destination.t.another_c (ie…
Rich
  • 15,602
  • 15
  • 79
  • 126
0
votes
0 answers

User without create table permission when using babelfish as a target endpoint in AWS DMS

I'm trying to migrate some data from SQL Server to AWS Aurora (Postgres) by using the babelfish capability. It was recently added as a target in the AWS DMS service. However, the connection test fails with the following message: Test Endpoint…
João Neto
  • 51
  • 1
  • 3
0
votes
1 answer

Data Migration from CSV (S3) to DynamoDB using AWS DMS

I am facing an issue when I am migrating a table consisting a JSON in a column which should be inserted in as a DyanmoDB Map. The source data is in the following format
VistaDev
  • 1
  • 1
0
votes
1 answer

AWS DMS Terraform Module Not Picking Up Kinesis Settings?

Description I'm hitting │ Error: 1 error occurred: │ * kinesis_settings must be set when engine_name = "kinesis" │ │ │ │ with module.replicate_oltp.aws_dms_endpoint.this["rates_kinesis_target"], │ on…
AlexLordThorsen
  • 8,057
  • 5
  • 48
  • 103
0
votes
1 answer

How AWS DMS CDC is working successfully without CDC On-premise MSSQL CDC prerequisites config?

We're using DMS for CDC Only migration for the time b/w point in time restore and current DB state, i.e AWS DMS to replicate changes as of the point in time at which you started your bulk load to bring and keep your source and target systems in…
Karikalan
  • 107
  • 10
0
votes
0 answers

Applying column filters in DMS Task for source MongoDB and target Postgres

In my AWS DMS Task Source - MongoDB Target - RDS Postgres SQL I am able to apply column filter and only 3 selected columns get migrated to target. But if the source mongo document contains an object datatype. So In the Include-column filter after…
Aim_headshot
  • 111
  • 1
  • 6
0
votes
0 answers

Mirroring homogeneous data from one MySQL RDS to another MySQL RDS

I have two MySQL RDS's (hosted on AWS). One of these RDS instances is my "production" RDS, and the other is my "performance" RDS. These RDS's have the same schema and tables. Once a year, we take a snapshot of the production RDS, and load it into…
Brian K
  • 548
  • 1
  • 4
  • 17
0
votes
1 answer

AWS DMS Replication task

How do I add a new table to an existing replication task and start CDC for it while not impacting/breaking the existing tables on going replication? Suppose Oracle to Oracle engine. I already have 4 tables in task and cdc is running and in sync.…
monto
  • 1
  • 1
0
votes
1 answer

Does AWS DMS has any way to touch or modify source tables?

I created several Full load, ongoing replication task DMS tasks to copy data from a VPS MySQL server to RDS MySQL, and after some tweaks (time zone config, collations, etc) I was able to get everything working as expected and replicating data…
Luis de Haro
  • 711
  • 1
  • 7
  • 28
0
votes
1 answer

AWS Data Migration Service Not Capturing Partitioned

Description I have a postgreSQL Partitioned Table shippo=> \d+ table_name Partitioned table "public.table_name" -- skipping column definitions Partitions: table_name_201909 FOR VALUES FROM…
AlexLordThorsen
  • 8,057
  • 5
  • 48
  • 103
1 2 3
…
30 31