Fetch AWS ReplicationGroup DNS using aws-cli

Question

We use cloudformation to deploy our service instances in aws. Part of our stack is currently an elasticache redis cluster, but we're trying to transition to a redis ReplicationGroup for a little extra redundancy.

We use ansible playbooks to automate the deployment of our service, including fetching things like Redis dns entries by fetching via aws-cli.

The problem is, aws-cli has the ability to look up a ReplicationGroup by id (aws-cli docs) but CloudFormation doesn't have the ability to actually set the id, instead the id is set to a random unique value:

Elasticache replication group id in CloudFormation template

So long story short, is there a single command I could use to query for my ReplicationGroup to get its primary DNS record?

Did you check this guide is that what you are looking for https://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/Replication.Endpoints.html#Replication.Endpoints.Finding.CLI — Piyush Patil, Aug 02 '16 at 18:35
Yes, I've looked there. That requires the ReplicationGroupId, which as I explained in my question I don't have. I only have the ReplicationGroup's description — StormeHawke, Aug 02 '16 at 18:37
I don't think that will be possible because Replication Groups are identified by there iD's using CLI and not there descriptions but don't know for sure if there is any other way. — Piyush Patil, Aug 02 '16 at 18:48

Matt Houser · Answer 1 · 2016-08-02T19:13:15.783

1

Your EC2 instances won't have a concept of a "current replication group". So, without knowing the replication group ID, you'll have to determine it yourself. There are a couple different ways of doing it:

Option 1:

In your CloudFormation template, pass the Replication Group ID to your EC2 instance via it's User Data.

Inside your CloudFormation template, you can use { "Ref" : "MyReplicationGroup" } to get the replication ID of the group to pass into the User Data.

Or you can use { "Fn::GetAtt" : [ "MyReplicationGroup", "PrimaryEndPoint.Address" ] } to get the primary endpoint's address.

If you're not otherwise using the User Data field of your launch configuration, you can simply reference the ID or endpoint address for the property:

"Properties" : {
  "UserData" : { "Ref" : "MyReplicationGroup" },
}

then inside the EC2 instance, grab the user data from the instance meta-data:

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html#instancedata-user-data-retrieval

$ curl http://169.254.169.254/latest/user-data

Option 2:

This is not a single command, but you can "find" the replication group:

From instance meta-data, get the Instance ID
Use AWS CLI describe-instances command to get the tags for your instance.
Amongst the tags will be a aws:cloudformation:stack-name tag which will identify your CloudFormation stack.
Use the AWS CLI describe-stack-resources command to get your stack's resources. In there, find your replication group ID from it's logical CloudFormation ID.

edited Aug 02 '16 at 19:13

answered Aug 02 '16 at 18:54

Matt Houser

33,983
6
70
88

Interesting. Can you elaborate a little bit on option 1? How would I pass it, and how would I access the value from the ec2 instance? – StormeHawke Aug 02 '16 at 18:57
Within the Launch Configuration of your EC2 instances, there is a `UserData` property. That property passes data into the EC2 instance. It can be data, or it can be a script. Currently, it's probably running a script to bootstrap Ansible. If so, you'll need to pass in the replication group as well (somehow). Reading it inside the EC2 instance will depend on how you're writing it into the User Data. – Matt Houser Aug 02 '16 at 19:03
Our build process uses ssh to kick off ansible, among some other business logic related things, we're not currently using userData at all at the moment – StormeHawke Aug 02 '16 at 19:11
So it looks like it should be the most basic case of populate the userdata and read it. Do I just add a userData property to my ec2 instance and add a ref to my replication group? Or how would I narrow it down to just the id? – StormeHawke Aug 02 '16 at 19:12
Turns out I was wrong about not using the userData field to kick off our ansible build. But I did find an option 3 that does what I want in what I think is a cleaner manner than option 2 even though it's similar in concept. Thanks for your help! – StormeHawke Aug 03 '16 at 17:05

score 0 · Accepted Answer · answered Aug 03 '16 at 17:03

After much digging and discussion with our devops team I found a way to do this that'll work with a single aws-cli command.

First, add an Outputs block to your redis configuration, on the same level as (outside of) the "Resources" block:

"Outputs" : {
        "RedisReplicationGroupDnsName" : {
            "Description" : "DNS entry for the redis replication group",
            "Value" : { "Fn::GetAtt" : [ "<your replication group's json key>", "PrimaryEndPoint.Address" ] }
        }
    },

Next, you'll need to set up an IAM policy that allows your instance to describe its own stack.

Finally, using aws-cli you can get the primary dns endpoint for the replication group with this command:

aws cloudformation describe-stacks --region {{ ansible_ec2_placement_region }} --stack-name <your stack name> --query 'Stacks[0].Outputs[?OutputKey==`RedisReplicationGroupDnsName`]|[0].OutputValue' --output text

Fetch AWS ReplicationGroup DNS using aws-cli

2 Answers2