Aurora Serverless password rotation setup using CloudFormation (and Lambda rotation templates)

Question

AWS has Fully Configured and Ready-to-Use Rotation Support for some supported RDS engines, including Amazon Aurora (Serverless also?)

I'm trying to setup the password rotation in my CloudFormation template using AWS::SecretsManager::RotationSchedule (note that this is not a fully functional template, only an illustration):

  DBCluster:
    Type: AWS::RDS::DBCluster
    Properties:
      Engine        : aurora
      EngineMode    : serverless
      EngineVersion : 5.6.10a

  Secret:
    Type: AWS::SecretsManager::Secret
    Properties:
      GenerateSecretString:
        SecretStringTemplate: '{"username": "admin"}'
        GenerateStringKey: password
        PasswordLength: 20
        ExcludeCharacters: '"@/\'

  SecretTargetAttachment:
    Type: AWS::SecretsManager::SecretTargetAttachment
    Properties:
      SecretId: !Ref Secret
      TargetId: !Ref DBCluster
      TargetType: AWS::RDS::DBCluster

  SecretRotation:
    Type: AWS::SecretsManager::RotationSchedule
    Properties:
      SecretId: !Ref UserAdminSecret
      RotationLambdaARN: <ARN_GET_FROM_SERVERLESS_APPLICATION_REPOSITORY>
      RotationRules:
        AutomaticallyAfterDays: 1

But the AWS Lambda rotation function fails with the following message:

"Database engine must be set to 'mysql' in order to use this rotation lambda": KeyError

Looks like Aurora Serverless is not supported by the AWS Lambda rotation function provided by AWS.

Is there an easy way to setup Aurora Serverless secret rotation using existing Lambda rotation templates?

Any example available to write my own rotation function for Aurora Serverless?

PS: This question is kind of related to Creating an Aurora Serverless Cluster from cloudformation?

score 2 · Answer 1 · answered Aug 07 '19 at 23:03

2

The RotationSchedule resource has a dependency on the SecretTargetAttachment resource. The attachment resource updates your secret-string value to contain connection information such as db engine, port and endpoint.

Unfortunately, there is no way for CloudFormation to know about this implicit dependency between the two resources. You need to put a DependsOn on the RotationSchedule resource with the attachment resource's logical id.

See the RotationSchedule resource in this example - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-secretsmanager-rotationschedule.html#aws-resource-secretsmanager-rotationschedule--examples

answered Aug 07 '19 at 23:03

Parimal

316
1
6

1

I don't know why you get a +1 this is not answering the question about Aurora Serverless – Yves M. Aug 08 '19 at 07:02
What you have is correct. Add this line to your RotationSchedule resource - "DependsOn: SecretTargetAttachment" – Parimal Aug 08 '19 at 22:26
@YvesM. Hi I'm having a similar issue now, my db engine is also Aurora MySQL, just wondering if you have solved this issue, can you share the function template? Thanks in advance. – wawawa Jan 29 '20 at 13:00
@Cecilia I haven't sold the problem yet, my password rotation is disabled. But I will investigate to craft a solution on day. Let me know if you find a ready-to-use solution – Yves M. Jan 29 '20 at 16:24
1

@YvesM.I actually found a potential solution, cracking on it at the moment, it should work if you are also using Aurora MySQL engine, we can just use the rotation function template for MySQL db, so instead of using CloudFormation, we can create a function manually (check this template https://github.com/aws-samples/aws-secrets-manager-rotation-lambdas/blob/master/SecretsManagerRDSMySQLRotationSingleUser/lambda_function.py), then tell Secretc Manager to call this function to perform rotation, Iet me do some experiment first. – wawawa Jan 29 '20 at 16:29

score 1 · Answer 2 · edited Nov 07 '21 at 13:20

I wasn't using serverless, but I got the same exact error.

"Database engine must be set to 'mysql' in order to use this rotation lambda": KeyError

The Solution

For me, the problem was that I needed to provide subnets and a security group to the rotation lambda.

A sketch of the CloudFormation template would look like (notice the parameters passed into the lambda):

DBSecrets:
  Type: AWS::SecretsManager::Secret
  Properties:
    GenerateSecretString:
      SecretStringTemplate: '{"username": "XXXXXXXXXX"}'
      GenerateStringKey: password
      PasswordLength: 24
      ExcludeCharacters: '"@/\'

DBSecretsRDSAttachment:
  Type: AWS::SecretsManager::SecretTargetAttachment
  Properties:
    SecretId: !Ref DBSecrets
    TargetId: !Ref RDSDatabase
    TargetType: AWS::RDS::DBInstance

SecretRotationSchedule:
  Type: AWS::SecretsManager::RotationSchedule
  DependsOn: DBSecretsRDSAttachment
  Properties:
    SecretId: !Ref DBSecrets
    RotationLambdaARN: !GetAtt MySQLRotationLambda.Outputs.RotationLambdaARN
    RotationRules:
      AutomaticallyAfterDays: 30

MySQLRotationLambda:
  Type: AWS::Serverless::Application
  Properties:
    Location:
      ApplicationId: <ARN_GET_FROM_SERVERLESS_APPLICATION_REPOSITORY>
      SemanticVersion: 1.1.0
    Parameters:
      endpoint: !Sub 'https://secretsmanager.${AWS::Region}.amazonaws.com'
      functionName: <Function Name>
      vpcSubnetIds: <Comma delimited List of VPC subnet IDs>
      vpcSecurityGroupIds: <Comma delimited List of VPC security grouop IDs>

RDSDatabase:
  Type: AWS::RDS::DBInstance
  Properties:
    MasterUsername: !Sub '{{resolve:secretsmanager:${DBSecrets}::username}}'
    MasterUserPassword: !Sub '{{resolve:secretsmanager:${DBSecrets}::password}}'
    Engine: mysql
    DBSubnetGroupName: <Your Subnet Group>
    VPCSecurityGroups: <Your Security Group>

Why is this the error that shows?

The rotation Lambda goes through the following steps:

First try to login with the pending secret, if it succeeds, return
Now try the current password
If both current and pending do not work, try previous

It fails to login with pending and current secret, then fails with this error when trying the previous secret. The pending and current secret are valid, the Lambda just can't make a connection to the database. The previous secret is the secret that you initially provide in the CloudFormation template above.

{
  "username": "XXXXXXXXXX", 
  "password": "XXXXXXXXXX"
}

The AWS::SecretsManager::SecretTargetAttachment changes it to the proper format (for RDS MySQL Single User):

{
  "engine": "mysql",
  "host": "<required: instance host name/resolvable DNS name>",
  "username": "<required: username>",
  "password": "<required: password>",
  "dbname": "<optional: database name. If not specified, defaults to None>",
  "port": "<optional: TCP port number. If not specified, defaults to 3306>"
}

The rotation Lambda nested stack has more parameters that you can pass in, just look at its template in the CloudFormation dashboard.

score 0 · Answer 3 · edited Apr 21 '20 at 00:52

0

I faced a similar error when setting the PostgreSQL parameters "password_encryption: 'scram-sha-256'"

The solution was drop entire CloudFormation stack recreate with MD5. (Updating the value did not resolve the error)

Also, if Lambdalog has timeout with no other errors, increase Lambda function timeout default 30 seconds to 60 seconds should resolve the issue.

edited Apr 21 '20 at 00:52

Bhargav Rao

50,140
28
121
140

answered Feb 04 '20 at 22:40

internetuser2008

1
1

1

It looks more like comment then an answer. Once you have sufficient [reputation](https://stackoverflow.com/help/whats-reputation) you will be able to [comment on any post](https://meta.stackexchange.com/questions/214173/why-do-i-need-50-reputation-to-comment-what-can-i-do-instead) – Anna Feb 05 '20 at 00:12

score 0 · Answer 4 · edited Nov 07 '21 at 13:18

I was able to setup secrets rotation for Aurora Serverless using AWS Fully Configured and Ready-to-Use Rotation Support: aws-secrets-manager-rotation-lambdas/SecretsManagerRDSPostgreSQLRotationSingleUser/

I was getting the same error mentioned in the Q above, I found out that in my Secrets setting the "engine": "postgres" setting was missing. After adding the setting as below it started working

{
  "username": "XXXX",
  "password": "XXXXXXXXXX",
  "engine": "postgres",
  "host": "db.cluster-XXXX.us-XXXX-X.rds.amazonaws.com",
  "port": 5432,
  "dbClusterIdentifier": "XXXXX"
}

Aurora Serverless password rotation setup using CloudFormation (and Lambda rotation templates)

4 Answers4

The Solution

Why is this the error that shows?

Linked