3

I've created a CloudFormation script that stands up a clone of our existing AWS stack. I've added in information so that CloudFormation will create RDS from either a snapshot of an existing RDS instance, or create a new instance with a named database. However, when I try to apply this same script to our existing stack (verified by creating a new stack with the script from the existing stack and then attempting to upgrade via the new script), CloudFormation always creates a new RDS instance.

Extracted portions of the CloudFormation script are below.

{
    "Parameters": {
        "liveDbName" : {
            "Default": "default",
                "Description" : "The live database name to create by default",
                "Type": "String",
                "MinLength": "1",
                "MaxLength": "64",
                "AllowedPattern" : "[a-zA-Z][a-zA-Z0-9]*",
                "ConstraintDescription" : "must begin with a letter and contain only alphanumeric characters."
        },
        "liveDbSnapshotIdentifier" : {
            "Description" : "This overrides Default Production Db name",
            "Type": "String",
            "Default": ""
        },
    },
    "Metadata": {
        "AWS::CloudFormation::Interface": {
            "ParameterGroups": [
                {
                    "Label": {
                        "default": "Db Layer Configuration"
                    },
                    "Parameters": [
                        "webDbInstanceType",
                        "liveDbName",
                        "liveDbSnapshotIdentifier",
                        "dbTimeZone",
                        "dbMasterUser",
                        "dbMasterPassword"
                    ]
                }
            ]
        }
    },
    "Conditions": {
        "UseLiveDbSnapshot" : { "Fn::Not" : [{ "Fn::Equals" : [ {"Ref" : "liveDbSnapshotIdentifier"}, "" ] }] },
    }
    "Resources": {
        "WebDb": {
            "Type": "AWS::RDS::DBInstance",
            "DeletionPolicy": "Snapshot",
            "Properties": {
                "AllocatedStorage": "100",
                "AutoMinorVersionUpgrade": "true",
                "BackupRetentionPeriod": "30",
                "CopyTagsToSnapshot": "true",
                "DBName" : {
                    "Fn::If" : [ "UseLiveDbSnapshot", { "Ref" : "AWS::NoValue"}, { "Ref" : "liveDbName" } ]
                },
                "DBSnapshotIdentifier" : {
                    "Fn::If" : [ "UseLiveDbSnapshot", { "Ref" : "liveDbSnapshotIdentifier" }, { "Ref" : "AWS::NoValue"} ]
                },
                "DBInstanceClass": {
                    "Ref": "webDbInstanceType"
                },
                "DBParameterGroupName": {
                    "Ref": "WebDbParameterGroup"
                },
                "DBSubnetGroupName": {
                    "Ref": "DbSubnetGroup"
                },
                "Engine": "mysql",
                "MasterUsername": {
                    "Ref": "dbMasterUser"
                },
                "MasterUserPassword": {
                    "Ref": "dbMasterPassword"
                },
                "MultiAZ": "true",
                "PubliclyAccessible": "false",
                "StorageType": "gp2",
                "Tags": [
                    {
                        "Key": "Name",
                        "Value": "WebDb"
                    }
                ]
            }
        }
    }
}

There are of course other portions of the script, but this is the portion (fully "namespaced," I believe) that deals with our database section.

What am I doing wrong with my script, and is there a correct way to do so? Obviously I don't want CloudFormation to restore a snapshot over our existing instance, but I don't want it to create a new instance with the named database, either.

EDIT: "Existing" stack script included

I've added the existing stack script as a link to Dropbox because the file is too long to include here directly: https://www.dropbox.com/s/313kmcnzk0pvyqi/sanitized-cloudformation.json?dl=0

Glen Solsberry
  • 1,536
  • 5
  • 28
  • 38
  • Can you provide your "existing" stack's template also? – Matt Houser Oct 18 '17 at 15:16
  • @MattHouser I'd rather not since it's about 9k lines and would take quite some time to sanitize. I can tell you that the only "real" difference between it and the version I'm using relates to the `Resources` for `WebDb`, specifically the `DBName` and `DBSnapshotIdentifier`; those don't exist at all in the existing stack template. If you believe it necessary, though, I'll sanitize and provide. – Glen Solsberry Oct 18 '17 at 15:25
  • It is difficult to diagnose an update without seeing both the before and after. – Matt Houser Oct 18 '17 at 15:26
  • @MattHouser please note that the only real addition in my script above vs the dropbox link is the addition of `DBName` and `DBSnapshotIdentifier`, and their use within the script. – Glen Solsberry Oct 18 '17 at 20:34

1 Answers1

1

Your original CloudFormation template did not include DBName or DBSnapshotIdentifier properties. So the RDS instance was created without a DBName. Any database housed by your RDS instance was created after-the-fact, and not by CloudFormation.

Your new template includes either the DBName or DBSnapshotIdentifier, depending on the input parameters.

According to the CloudFormation reference docs for the AWS::RDS::DBInstance resource, if you add/change/remove either of the DBName or DBSnapshotIdentifier properties, then the RDS instance will be re-created.

Source: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-rds-database-instance.html

I'm guessing that when you're applying the updated template, you're attempting to use the name of your database that you have inside your RDS instance as the value for liveDbName. However, as far as CloudFormation is concerned, this is a change to the RDS instance and requires replacement.

To apply the template update, you'll need to modify it such that neither DBName nor DBSnapshotIdentifier are applied.

Matt Houser
  • 10,053
  • 1
  • 28
  • 28
  • So you're suggesting adding the `DBName`/`DBSnapshotIdentifier` fields with `AWS::NoValue`, upgrading the stack, and then adding them with the full "logic," and upgrading again? – Glen Solsberry Oct 19 '17 at 13:02
  • @GlenSolsberry No. The moment you include a value for `DBName` or `DBSnapshotIdentifier`, your RDS instance will be re-created. To avoid that, you must forever avoid setting those properties on your existing stack. The logic is OK to be there, but all future updates on your existing stack must resolve to `AWS::NoValue` or be absent. So you must adjust your logic to set one, the other, or neither, and the neither case is the one you must apply to this stack going forward. – Matt Houser Oct 19 '17 at 13:19
  • Any suggestions on how to actually do that in my script? Would I basically just nest `Fn::If` to check for the existence of either, and if neither were set, use `AWS::NoValue`? – Glen Solsberry Oct 19 '17 at 13:32
  • I think you'd need 2 conditions, one for the `DBName`, and one for the `DBSnapshotIdentifier`. You could keep the `Fn::If` independent, but you'd end up with the case of specifying both if both inputs were specified. To avoid that, you could try nested `Fn::If`. – Matt Houser Oct 19 '17 at 13:46