How to retrieve Secret Manager data in buildspec.yaml

Question

Im working on creating the CodeBuild which is integrated with SonarQube, So I pass values and sonar credentials directly in my Buildspec.yaml

Instead of Hardcoding directly, I tried to retrieve using the below command from SecretManager as it is mentioned in the below link. But it is not getting the correct values. it throws an error.

Command : '{{resolve:secretsmanager:MyRDSSecret:SecretString:username}}'

Link: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html#dynamic-references-secretsmanager

Error [ERROR] SonarQube server [{{resolve:secretsmanager:arn:aws:secretsmanager:us-east-1:********:secret:**********:SecretString:SonarURL}}] can not be reached

How I used echo '{{resolve:secretsmanager:arn:aws:secretsmanager:us-east-1:***:secret:**************:SecretString:*******}}'

Note: All the * inside my commard are the secretname and secreturl

score 15 · Answer 1 · answered Nov 06 '19 at 23:42

15

CodeBuild just launched this today - https://aws.amazon.com/about-aws/whats-new/2019/11/aws-codebuild-adds-support-for-aws-secrets-manager/

answered Nov 06 '19 at 23:42

committedandroider

8,711
14
71
126

1

This should be the accepted answer. It is the most direct solution to op's question. – Arik Sep 23 '21 at 18:39
1

Yes, this is the answer. https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#secrets-manager-build-spec In the following example, TestSecret is the name of the key-value pair stored in Secrets Manager. The key for TestSecret is MY_SECRET_VAR. You access the variable during the build using the LOCAL_SECRET_VAR name. env: secrets-manager: LOCAL_SECRET_VAR: "TestSecret:MY_SECRET_VAR" – Franxi Hidro Dec 14 '21 at 13:52

score 12 · Accepted Answer · edited Dec 29 '20 at 19:51

12

If you wish to retrieve secrets in your buildspec file, I would recommend to use Systems Manager Parameter Store which is natively integrated with CodeBuild. Systems Manager is a service in itself, search it from the AWS Console homepage, then Paramater Store is in the bottom left of the Systems Manager Console page.

Lets assume you want to include Access Key and Secret Key in buildspec.yml file:
- Create AccessKey/SecretKey pair for a IAM User
- Save the above keys in an SSM parameter store as secure string (e.g. '/CodeBuild/AWS_ACCESS_KEY_ID' and '/CodeBuild/AWS_SECRET_ACCESS_KEY')
- Export the two values in your build environment using the following buildspec directive(s):

version: 0.2
env:
    parameter-store:
        AWS_ACCESS_KEY_ID_PARAM: /CodeBuild/AWS_ACCESS_KEY_ID
        AWS_SECRET_ACCESS_KEY_PARAM: /CodeBuild/AWS_SECRET_ACCESS_KEY

phases:
    build:
        commands:
            - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID_PARAM
            - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY_PARAM
            # Your Ansible commands below
            - ansible-playbook -i hosts ec2-key.yml

[1] Build Specification Reference for CodeBuild - Build Spec Syntax - https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html#build-spec-ref-syntax

edited Dec 29 '20 at 19:51

Beachhouse

4,972
3
25
39

answered Oct 03 '19 at 13:43

shariqmaws

8,152
1
16
35

Thanks for this quick turn around and help me in getting some idea on Parameter Store. – Sajiv Sriraam Oct 04 '19 at 10:54
1

Can you give a sample on how can it be used in CodeBuild & CodePipeline? for buildspec.yaml parameter-store: CUSTOM_PARAM: /aws/reference/secretsmanager/DBUser1 Eg: I have a Secret named "SajivSecret" which has few key and value pair. In which one Key named "DBUser1" and Value as some "***". I want to retrive this specific key in my buildspec.yaml file My Question is, Will it look for the Secret Named DBUSer1 or Key as DBUser1 in all the secrets in Secret Manager – Sajiv Sriraam Oct 04 '19 at 11:02
How to retrieve secrets specific to my Secret in Secret Manager in buildspec.yaml under parameter-store – Sajiv Sriraam Oct 04 '19 at 11:12
@SajivSriraam It will look for the Secret Named DBUSer1 - "When you retrieve a Secrets Manager secret from Parameter Store, the parameter name must begin with the following reserved path: /aws/reference/secretsmanager/secret_ID_in_Secrets_Manager" - https://docs.aws.amazon.com/systems-manager/latest/userguide/integration-ps-secretsmanager.html – committedandroider Oct 07 '19 at 01:04
@committedandroider - Where can i find the secret ID in secret Manager. I could see SecretKey Name, Secret Name, Secret ARN – Sajiv Sriraam Oct 07 '19 at 06:32
@SajivSriraam secret id is the secret name – committedandroider Oct 07 '19 at 21:20
okay.. then how to retrive the particular Secret that is stored inside the Secret? – Sajiv Sriraam Oct 08 '19 at 10:29
aws secretsmanager get-secret-value --secret-id, then pass in the name of the secret – committedandroider Nov 06 '19 at 23:46

score 5 · Answer 3 · answered Oct 03 '19 at 17:15

The dynamic reference syntax you are trying to use only works with the Cloud Formation (CFN) service. In some cases, CFN restricts where these dynamic references to secrets will expand. Specifically, they do not expand in places where the secrets might be visible in the console, such as in EC2 metadata.

If you are trying to setup Code Build via CFN, this may be what you are seeing. However, as shariqmaws mentioned, you can use parameter store and either store your secret there or use parameter store as a pass through to secrets manager (in case you want to use secrets manager to rotate your secrets or for other reasons).

Can you give a sample on how can it be used in CodeBuild & CodePipeline? for buildspec.yaml parameter-store: CUSTOM_PARAM: /aws/reference/secretsmanager/DBUser1 Eg: I have a Secret named "SajivSecret" which has few key and value pair. In which one Key named "DBUser1" and Value as some "***". I want to retrive this specific key in my buildspec.yaml file My Question is, Will it look for the Secret Named DBUSer1 or Key as DBUser1 in all the secrets in Secret Manager — Sajiv Sriraam, Oct 04 '19 at 14:43

score 0 · Answer 4 · answered Jul 26 '22 at 16:26

version: 0.2

env:
  parameter-store:
    AWS_ACCESS_KEY_ID         : /terraform-cicd/AWS_ACCESS_KEY_ID
    AWS_SECRET_ACCESS_KEY     : /terraform-cicd/AWS_SECRET_ACCESS_KEY
    AWS_CODECOMMIT_SSH_ID     : /terraform-cicd/AWS_CODECOMMIT_SSH_ID
  secrets-manager:  
    AWS_CODECOMMIT_SSH_PRIVATE: /terraform-cicd/AWS_CODECOMMIT_SSH_PRIVATE

How to retrieve Secret Manager data in buildspec.yaml

4 Answers4

Linked