Stack Overflow
Stack Overflow

Questions tagged [aws-dms]

AWS Database Migration Service (AWS DMS) can migrate your data to and from most widely used commercial and open-source databases such as Oracle, PostgreSQL, Microsoft SQL Server, Amazon Redshift, Amazon Aurora, MariaDB, and MySQL. The service supports homogeneous migrations such as Oracle to Oracle, and also heterogeneous migrations between different database platforms, such as Oracle to MySQL. The source or target database must be on an AWS service.

http://docs.aws.amazon.com/dms/latest/userguide/Welcome.html

465 questions
5
votes
2 answers

AWS DMS Tasks Will Not Delete

I am trying to delete two AWS DMS database "migration" tasks that are in the failed state. After over 30 tries from the console, where it shows a green banner and displays that the tasks are deleted - the tasks are still there. I also attempted to…
drobin
  • 286
  • 2
  • 6
5
votes
1 answer

Can SnowFlake be used as a source endpoint in Data Migration Service of AWS?

I am trying to use AWS DMS Database Migration Service along with Snowflake as a source database. Is there any way I can achieve this ? All I could see options for IBMDB2, MySQL, SQL Server, Amazon Aurora, Oracle, SAP Sybase etc. But not for…
Aniket Maithani
  • 845
  • 1
  • 10
  • 24
5
votes
3 answers

AWS DMS SQL Server to s3 parquet - change-data-type transformation rule and 'Parquet type not supported: INT32 (UINT_8)'

We use AWS DMS to dump SQL Server DBs into S3 as parquet files. Idea is to run some analytics with spark over parquets. When a full load is complete then it's not possible to read parquets since they have UINT fields in the schema. Spark declines to…
Anton
  • 1,432
  • 13
  • 17
5
votes
2 answers

AWS DMS - Microsecond precision for CDC on MYSQL as source EndPoint

I am using AWS DMS for migrating data from MYSQL as source endpoint and S3 as target endpoint. I want to track the updates from source so during the configuration, I have enabled TimestampColumnName property (col name : event_timestamp). In the…
BlackJack
  • 139
  • 3
  • 14
5
votes
1 answer

AWS DMS (Database Migration Services) full LOB not working for SQL Server

I'm trying to migrate a SQL Server table using AWS DMS to a DynamoDb target. The table structure is as follows: |SourceTableID|Title |Status|Display|LongDescription| |-------------|-----------|------|-------|---------------| |VARCHAR(100)…
luisgepeto
  • 763
  • 1
  • 11
  • 36
5
votes
1 answer

Amazon DMS task with custom rules fails when sinking to Kinesis

I'm trying to listen to Aurora DB changes using Amazon DMS and push the changes to a Kinesis stream, where a Lambda function listening to the stream will do the processing. I was referring to the below documentation to write my…
Praneeth Peiris
  • 2,008
  • 20
  • 40
5
votes
0 answers

AWS DMS - Migrate array from MongoDB to DynamoDB

I'm trying to migrate data from MongoDB to DynamoDB. Based on my test, it seems DMS can only convert array into string. For example: { data: ['one', 'two', 'three']} It will be { data: { 'S': [\"one\", \"two\", \"three\"] } } Is it possible to…
zeck
  • 769
  • 1
  • 7
  • 13
5
votes
0 answers

Merge data from multiple tables in RDS to export to Redshift using AWS DMS

I am setting up a continuous replication from RDS to Redshift using AWS DMS, as suggested here. My usecase requires me to combine data from multiple RDS tables to export to Redshift. For eg. RDS has Tables School, Student, District. I want to export…
Adi
  • 387
  • 3
  • 6
  • 14
4
votes
0 answers

AWS DMS for CDC - Avoid creating new databases/tables

We need to sync a users table on one database instance (MySQL) to a users table on another database instance (Aurora/MySQL 8) in real time by using CDC (change data capture). We use multitenancy, so we have a ton of schemas (databases) on the source…
Mike Furlender
  • 3,869
  • 5
  • 47
  • 75
4
votes
1 answer

AWS - migration failed with error Value of column '' in table '' was truncated to 32768 bytes

I am using the AWS DMS service to migrate my Postgres database to Aurora Postgres but I got in some tables that error Value of column 'ColumnName' in table 'TableName' was truncated to 32768 bytes, actual length: 110644 bytes column type could be…
Mina Fawzy
  • 20,852
  • 17
  • 133
  • 156
4
votes
2 answers

Connection fails when adding ElasticSearch as a Target for AWS Data Migration Service

I'm trying to set up ElasticSearch endpoint in DMS but no matter what I try when I test the connection I get this generic error: Test Endpoint failed: Application-Status: 1020912, Application-Message: Endpoint initialization failed. I've created a…
Adam Cooper
  • 8,077
  • 2
  • 33
  • 51
4
votes
1 answer

Can i autoscale DMS Replication instance?

I Have created DMS with replication instance and mysql to mysql endpoints. My replication instance will be running 24*7. I will run run mysql to mysql tasks frequently on it. But the size of data will always be varying. I want my DMS to be…
Joseph N
  • 540
  • 8
  • 28
4
votes
0 answers

AWS DMS: Error when migrating from RDS SQL Server to Kinesis Streams

I'm facing an issue when using DMS to replicate data from a SQL Server (RDS) to Kinesis Streams. I'm replicating ~20 tables (~1TB) in an environment A and the replication is only ON GOING (through CDC), not full load. The process goes fine until it…
Flavio N
  • 41
  • 3
4
votes
1 answer

How to keep column names using AWS DMS with S3 as a target?

After the full load using AWS DMS, the CSV file doesn't contain the colunm names, how can I keep the column names as a part of the CSV file?
Luan Kevin Ferreira
  • 1,184
  • 2
  • 16
  • 40
4
votes
2 answers

Not able to import certificate as documented

I followed all steps mentioned on this page. https://docs.aws.amazon.com/dms/latest/userguide/target.docdb.html The problem is that the "import certificate" button is disabled and I can not upload the certificate. You can download the public key…
shantanuo
  • 31,689
  • 78
  • 245
  • 403
1
2
3
…
30 31