Stack Overflow
Stack Overflow

Questions tagged [aws-dms]

AWS Database Migration Service (AWS DMS) can migrate your data to and from most widely used commercial and open-source databases such as Oracle, PostgreSQL, Microsoft SQL Server, Amazon Redshift, Amazon Aurora, MariaDB, and MySQL. The service supports homogeneous migrations such as Oracle to Oracle, and also heterogeneous migrations between different database platforms, such as Oracle to MySQL. The source or target database must be on an AWS service.

http://docs.aws.amazon.com/dms/latest/userguide/Welcome.html

465 questions
3
votes
0 answers

AWS DMS LOB Support (Relational to ES)

I have Full LOB enabled for a DMS migration but still get the following error: Column 'data_json' is unsupported in table def 'table.users' since the LOB support is disabled. Is this a bug in DMS or is my configuration incorrect?. The data_json…
Artem Kalinchuk
  • 6,502
  • 7
  • 43
  • 57
3
votes
0 answers

AWS DMS data transformation

I see that we can create new columns during a DMS task using "rule-action=add-column" and "expression" to define the content of the new column. But what if we want to transform the data of an existent column? Example: I want to encrypt a column…
user13520400
  • 161
  • 1
  • 5
3
votes
0 answers

AWS DMS ( Data migration service ) - Replication is stopped without error

We have recently observed that the DMS Task stops reading the archive logs suddenly, with no error. Source : Oracle Database. Logs are been read from ASM. Target : S3 bucket.  17:33:36 2020-05-12T17:33:36 [SORTER ]I: Task is running…
Vishal Sivala
  • 35
  • 4
3
votes
2 answers

AWS DMS CDC task does not detect column name and type changes

I have created a CDC task that captures changes in a source PostgreSQL schema and writes them in Parquet format into a target S3 bucket. The task captures the inserts, updates and deletes correctly but fails to capture column name and type changes…
SaadK
  • 256
  • 2
  • 10
3
votes
1 answer

Can we use AWS Data Migration Service for replication from Aurora Serverless as source?

My DMS replication instance (which is in same VPC as of Aurora serverless DB instance) is not able to find DB while creating endpoint in DMS. However, I am able to create a cloud9 instance in same VPC as aurora serverless instance and connect to it…
conetfun
  • 1,605
  • 4
  • 17
  • 38
3
votes
2 answers

dms s3 source endpoint connection fails

Getting below connection error when trying to validate S3 source endpoint of DMS. Test Endpoint failed: Application-Status: 1020912, Application-Message: Failed to connect to database. Followed all the steps listed in the below links but still…
nmakb
  • 1,069
  • 1
  • 17
  • 35
3
votes
2 answers

How to fix `column "xlog_position" does not exist` error when using AWS DMS for Postgres to Postgres data migration

I'm trying to migrate and synchronize a PostgreSQL database using AWS DMS and I'm getting the following error. Last Error Task error notification received from subtask 0, thread 0 [reptask/replicationtask.c:2673] [1020487] RetCode: "SQL_ERROR…
Isuru Pathirana
  • 1,060
  • 1
  • 16
  • 37
3
votes
2 answers

AWS DMS - Error - "AWS Account owning the Role must match the caller of the API"

I am trying to create a destination in AWS DMS for an S3 and I am getting this message "AWS Account owning the Role must match the caller of the API", I searched for this error and could not find what it refers to. Any idea what I have to do?
user3661384
  • 524
  • 7
  • 18
3
votes
2 answers

AWS DMS RDS MySQL 8 Can't initialize character set unknown

I am trying to run a replication task (CDC) between a AWS RDS MYSQL 8.0.11 database and a Redshift cluster using DMS. I run into the error : Error 2019 (Can't initialize character set unknown (path: compiled_in)) connecting to MySQL…
Hugo
  • 1,195
  • 2
  • 12
  • 36
3
votes
1 answer

AWS DMS: How to handle TIMESTAMP_MICROS parquet fields in Presto/Athena

Presently, we have a DMS task that will take the contents of a MySQL DB and dump files to S3 in parquet format. The format for the timestamps in parquet ends up being TIMESTAMP_MICROS. This is a problem as Presto (the underlying implementation of…
JD D
  • 7,398
  • 2
  • 34
  • 53
3
votes
1 answer

Can I pause a AWS DMS replication instance?

I am using AWS DMS to move data around daily. My replication instances are not used 24/24 hours. Can I pause them in order to save on costs ? If I cannot pause them, would it be bad to create and destroy them daily ? Would it be too much work for…
Hugo
  • 1,195
  • 2
  • 12
  • 36
3
votes
1 answer

Cross Account DMS Replication for RDS instances behind Bastion machines

I have production stacks inside a Production account and development stacks inside a Development account. The stacks are identical and are setup as follows: Each stack as its own VPC. Within the VPC are two public subnets spanning to AZs and two…
user1638152
  • 577
  • 11
  • 23
3
votes
2 answers

AWS DMS Issue with boolean column

I'm trying to enable replication with DMS, using as source an Aurora mySQL instance and as destination a Redshift instance. The replication fails on boolean columns. I have declared the boolean column as BIT(1) on the mySQL instance. According to…
Spiff
  • 3,873
  • 4
  • 25
  • 50
3
votes
4 answers

How AWS DMS works internally

In AWS DMS how does the migration happening internally? Is it like exporting entire data from source table and importing to destination table? Or is it like migrating table records one by one to destination table? I am new to aws dms and don't have…
user2833608
  • 51
  • 1
  • 5
3
votes
1 answer

AWS DMS unable to truncate SQL transaction log

We are running AWS DMS with SQL as source and DynamoDB as destinations. We are are having difficultly truncating the transaction log, we see 2 SQL tasks per AWS DMS that look as follows: begin transaction [WVGLQ7HFWFWJCCPK4RQVRQGDVE]; update…
Haroon
  • 1,052
  • 13
  • 28
1 2 3
…
30 31