0

I'm using Amazon CloudFormation to describe my desired infrastructure in AWS cloud. In Amazon Secrets Manager I've created few variables, that are representing credentials for S3 as well as database access in my cluster. This secrets are injected into TaskDefinition like this:

Resources: {...}
    TaskDefinition: {...}
        Properties:
            ContainerDefinitions:
                -   Name: !Ref ServiceName
                    Image: {...}
                    Environment:
                        - 
                          Name: AWS_S3_SECRET_KEY
                          Value: 
                              Fn::Join:
                                    - ''
                                    -   - '{{resolve:secretsmanager:'
                                        -   Fn::ImportValue: !Ref S3CredentialsId
                                        - ':SecretString:aws_s3_secret}}'

The problem with this approach is, that the TaskDefinition is not fetching any changes to the Amazon Secrets Manager automatically and therefore don't update the revision of the TaskDefinition -> all recently started containers are still referencing old values of the secret. That is why if I update the value of e.g. the aws_s3_secret then this change will be not propagated to all of the existing and newly created containers in my Fargate cluster.

So far I've achieved this propagation with "dirty" approach - by adding some not existing environment variable to the TaskDefinition and applying this change to the cluster (I've uploaded the updated CloudFormation template of the service). Only after this action there was a new revision of the TaskDefinition created, that contained updated secrets value, as Fargate mentioned difference in the TaskDefinition description.

Therefore is my question: How to achieve this propagation of the secret changes into the TaskDefinition automatically?

ilja
  • 351
  • 2
  • 14

1 Answers1

3

This doesn't directly answer your question since as far as I know, it's not possible to force propagation of secret changes via CloudFormation.

That being said, you shouldn't be using Secrets Manager to manage AWS credentials used to access S3. Instead, use IAM Roles for Tasks (https://docs.aws.amazon.com/AmazonECS/latest/userguide/task-iam-roles.html). This attaches an IAM role to your task definition, which can then be used to call S3 or even STS AssumeRole if you need to assume a different role with permissions on the S3 bucket.

If you need to access other secrets, take a look at the following tutorial: https://docs.aws.amazon.com/AmazonECS/latest/userguide/specifying-sensitive-data-tutorial.html. By following it you should be able to specify the Secrets in the CloudFormation container definitions. Here's an example of specifying the secret using the Secrets container definition. I haven't had time to test if it works, but it should according to the AWS documentation.

Resources: {...}
    TaskDefinition: {...}
        Properties:
            ContainerDefinitions:
                -   Name: !Ref ServiceName
                    Image: {...}
                    Secrets:
                        -
                          Name: AWS_S3_SECRET_KEY
                          ValueFrom: arn:aws:secretsmanager:region:aws_account_id:secret:username_value-u9bH6K
ilja
  • 351
  • 2
  • 14
WillT
  • 109
  • 5
  • If you must access Secrets Manager from your task, see https://docs.aws.amazon.com/AmazonECS/latest/userguide/specifying-sensitive-data-tutorial.html – WillT Apr 14 '20 at 23:20
  • thank you for your hint! I'll reconsider my approach to save S3 credentials to the Secrets Manager. But still, besides from my S3 credentials I also have more secrets to store - like e.g. Database URL and credentials. So the main question is still relevant to me. I'll update the question later to point out this detail. – ilja Apr 15 '20 at 07:21
  • I linked a tutorial in my followup comment above. The tutorial allows you to specify a Secrets Manager ARN and an environment variable name in your task definition. ECS will pull from Secrets Manager and populate the environment variable for you when executing the task. You should be able to use this approach in your CloudFormation container definition instead. – WillT Apr 15 '20 at 16:21
  • Edited my answer to include an example CloudFormation template – WillT Apr 15 '20 at 16:31