AWS ECS - How to retrieve specific key from secret manager?

Question

I may be missing something obvious here, but I can't seem to find documentation about retrieving a specific key/value from a secrets manager secret for an ECS task definition.

I have a secret with an ARN like so...

arn:aws:secretsmanager:<region>:<account>:secret:LDAP_Bind_Credentials-abcd

Within this secret I have key/value pairs like so...

LDAP_BIND_USER: <ldap bind user name>
LDAP_BIND_PASSWORD: <ldap bind user password>

What I want to be able to do, is define the environment variables in my task definition LDAP_BIND_USER and LDAP_BIND_PASSWORD, and reference the appropriate key within my secret.

Is this actually possible, or am I supposed to actually do the decoding of the key/value pairs within my program?

The documentation only seems to reference the ARN of the secret itself, not the key/value pairs within the secret.

Answer 1

Since February 2020, ECS task definition now supports reading AWS Secrets Manager secrets from a key within a JSON object for tasks using the EC2 launch type.

You could add the following in the containerDefinitions of your task definition file

{
  "containerDefinitions": [{
    "secrets": [{
      "name": "<environment_variable_name>",
      "valueFrom": "arn:aws:secretsmanager:<region>:<account_id>:secret:<secret_name>:<json_key>::"
    }]
  }]
}

Reference: AWS ECS secret manager documentation

Answer 2

Using secret it is not possible and you have to do the decoding inside the program.

Generally when ECS will retrieve the secrets it will give us that in json object like {"admin":"admin","pas":"pas"} and we have to decode it programmatically.

But another alternative will be make use of the AWS System Manager parameter store [1] to store secrets and in that case ECS will give you directly actual value for it.

[1] https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-parameter-store.html

Answer 3

After some hours of try and error, i've came to the following Solution:

  DatabaseSecret:
    Type: "AWS::SecretsManager::Secret"
    Properties:
      Name: !Sub "application-${Stage}-databasesecret"
      Description: AWS RDS admin credentials
      GenerateSecretString:
        SecretStringTemplate: '{"POSTGRES_USER": "username"}'
        GenerateStringKey: POSTGRES_PASSWORD
        PasswordLength: 32
        ExcludeCharacters: '"@/\_[]'

This will generate a Secret having 2 values: POSTGRES_PASSWORD and POSTGRES_USER

  Database:
    Type: AWS::RDS::DBInstance
    Properties:
      Engine: postgres
      VPCSecurityGroups:
        - !Ref DatabaseSecurityGroup
      DBInstanceClass: db.t3.micro
      AllocatedStorage: "5"
      MasterUsername: !Sub "{{resolve:secretsmanager:${DatabaseSecret}::POSTGRES_USER}}"
      MasterUserPassword: !Sub "{{resolve:secretsmanager:${DatabaseSecret}::POSTGRES_PASSWORD}}"

This will apply the values to the RDS instance.

I was unable to address the Properties directly. But i was able to retrieve the whole Secret as JSON (having only the 2 secret strings)

  ApiTaskDefinition:
    Type: AWS::ECS::TaskDefinition
    UpdateReplacePolicy: Retain
    Properties:
      ...
      ContainerDefinitions:
        - Name: "some-name"
          Image: !Ref ImageName
          Memory: 512
          ...
          Environment:
            - Name: POSTGRES_HOST
              Value: !GetAtt Database.Endpoint.Address
            - Name: POSTGRES_PORT
              Value: !GetAtt Database.Endpoint.Port
            - Name: POSTGRES_DB
              Value: "postgres"
          Secrets:
            - Name: POSTGRES_CREDENTIALS
              ValueFrom: !Ref DatabaseSecret

This will set an environment variable to the ECS like e.g.

[{"POSTGRES_USER": "username", "POSTGRES_PASSWORD": "password"}]

You can then access these values by parsing the environment variable as JSON

Typescript example:

    const dbCredentialsConfig = this.configService.get('POSTGRES_CREDENTIALS');
    const dbCredentials = JSON.parse(dbCredentialsConfig)[0];

 return {
      type: 'postgres',
      host: this.configService.get('POSTGRES_HOST'),
      port: this.configService.get('POSTGRES_PORT'),
      username: dbCredentials.POSTGRES_USER,
      password: dbCredentials.POSTGRES_PASSWORD,
      database: this.configService.get('POSTGRES_DB'),
      ...
    };