Connection fails when adding ElasticSearch as a Target for AWS Data Migration Service

Question

I'm trying to set up ElasticSearch endpoint in DMS but no matter what I try when I test the connection I get this generic error:

Test Endpoint failed: Application-Status: 1020912, Application-Message: Endpoint initialization failed.

I've created a role trusted by dms.amazonaws.com:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "1",
      "Effect": "Allow",
      "Principal": {
        "Service": "dms.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

and given it every permission:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "*"
            ],
            "Resource": "*"
        }
    ]
}

I have also configured my Elasticsearch domain to be public and I can connect to it and access Kibana.

I followed along with this 'tutorial' but now I'm stuck as to where to look for logs/potential issues with Elastic.

Hey Adam, did you ever end up having any luck with this? One thing to note for my own debugging purposes (and anyone else interested), AWS will not let completely public access to an Elasticsearch domain, but I did add a rule s.t. it is accessible via certain IPs to confirm that the domain is running as expected. — ostrumvulpes, Apr 29 '21 at 16:41

score 3 · Accepted Answer · answered Sep 29 '21 at 11:45

3

It turns out that the cause here was a trailing slash in the Elastic url. I was setting the url as:

https://my-elastic-search-name-1lllsyiodfokjalksnd.eu-west-1.es.amazonaws.com/

which was failing when I switched to setting the url as:

https://my-elastic-search-name-1lllsyiodfokjalksnd.eu-west-1.es.amazonaws.com

Everything started to work Hopefully a bug that'll get fixed soon.

answered Sep 29 '21 at 11:45

Adam Cooper

8,077
2
33
51

1

I confirm that it worked to me as well :) – evgpisarchik Feb 27 '23 at 12:26

score 1 · Answer 2 · answered May 06 '21 at 21:29

1

I ran into the same issue, and had to modify the Elasticsearch domain's access policy to allow the role attached to my DMS target endpoint to access it.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::{accountNumber}:role/{serviceRoleUsedForDmsTargetEndpoint}"
      },
      "Action": [
        "es:ESHttpDelete",
        "es:ESHttpGet",
        "es:ESHttpHead",
        "es:ESHttpPost",
        "es:ESHttpPut"
      ],
      "Resource": "{yourElasticsearchDomainArn}"
    }
  ]
}

This wasn't a solution I found via the tutorials/documentation, but through a support case instead.

answered May 06 '21 at 21:29

ostrumvulpes

542
1
4
14

That makes sense, I’ll treat this when I get a chance – Adam Cooper May 07 '21 at 10:38
Unfortunately this didn't work for me, currently working through the issue with aws support who also are unable to identify the cause. – Adam Cooper Jun 24 '21 at 13:42
1

Same here. I'm sure that is needed, but I'm sure I'm missing something else. either a SG issue or an IAM issue. the AWS docs are completely worthless on this subject. – grayaii Sep 10 '21 at 19:50
2

@grayaii Finally tracked down the issue with aws, and it was caused by a trailing slah in the elastic url. – Adam Cooper Sep 29 '21 at 11:46

Connection fails when adding ElasticSearch as a Target for AWS Data Migration Service

2 Answers2