10

I'm trying to add an environment variable to a Pipeline action that uses AWS Codebuild. However, no matter what I add, if I choose a type of Secret Manager, the step fails with the following error:

Invalid action configuration

EnvironmentVariables specified in action config does not match expected format, the expected format is JSON array adhering to the following format: [{"name": "string", "type": "string", "value": "string"}]

This is what I'm entering in the UI:

enter image description here

And the JSON that CodePipeline is generating looks like this:

[{"name":"SERVICE_CREDS","value":"my-secret:service_creds","type":"SECRETS_MANAGER"}]

What is going on here?? I don't know what I could possibly be getting wrong on my end. I'm entering text into the boxes they provide. The JSON that Pipelines produces from they input boxes looks valid to my eye. So, I have no idea why it is saying that the environment variables aren't matching the expected format!

Community
  • 1
  • 1
in_rogers_neighborhood
  • 197
  • 1
  • 6
  • "Value" cannot have a ":" in it as you are putting there. It will result in invalid json under the hood. – shariqmaws Apr 12 '20 at 05:11
  • I'm not sure that that is the issue. The JSON produced is perfectly valid (this can be verified with https://jsonformatter.curiousconcept.com/). Additionally, the `:` are how you specify the Secret Manager path, so they cannot be dropped. – in_rogers_neighborhood Apr 12 '20 at 18:49
  • I have the same issue, did you found a solution? – boxi Jun 29 '20 at 00:32

2 Answers2

11

If anyone comes to this page after searching for the error:

EnvironmentVariables specified in action config does not match expected format, the expected format is JSON array adhering to the following format

This is a recurring issue when your have a CodePipeline which feeds an environment variable '#{SourceVariables.CommitMessage}' from Source action to CodeBuild action and if the CommitMessage contains a quote or is multi line, then the action will fail due to internal json parser failure.

Note: CodeCommit always adds a '\n' so this issue will always occur with CodeCommit. For GitHub, it will only occur if you use the extended commit message.

For now to workaround this issue without loosing the 'COMMIT_MESSAGE' environment variable, please follow these steps:

Workaround:

  • Remove the 'COMMIT_MESSAGE' Environment Variable from CodePipeline configuration on the CodeBuild action.

  • Make sure your CodeBuild project's service role has permission to do 'ListPipelineExecutions' on the Pipeline.

  • Add the following in Buildspec 'Install' phase to install 'jq' utility [1]:

    - apt-get install jq

  • Add the following in Buildspec where you need to get the commit message (please update to the name of the pipeline):

    - COMMIT_MESSAGE=$(aws codepipeline list-pipeline-executions  --pipeline-name <Pipeline_Name> --max-items 1 | jq -r '.pipelineExecutionSummaries[0].sourceRevisions[0].revisionSummary')
- export COMMIT_MESSAGE
- echo $COMMIT_MESSAGE # debug command only
- printenv # debug command only

Using this method, we are using the 'list-pipeline-executions' [2] AWS CLI call to retrieve the recent pipeline execution and parse the commit message from this execution. The 'COMMIT_MESSAGE' variable will include the complete commit message with any quotes or newlines.

References:

[1] jq - https://stedolan.github.io/jq/

[2] list-pipeline-executions - https://docs.aws.amazon.com/cli/latest/reference/codepipeline/list-pipeline-executions.html

shariqmaws
  • 8,152
  • 1
  • 16
  • 35
1

In continuation with the above answer, instead of using/installing jq in buildspec.yml, we can also use the AWS inbuilt query flag/method as shown below.

- COMMIT_MESSAGE=$(aws codepipeline list-pipeline-executions --pipeline-name <Pipeline_Name> --max-items 1 --query 'pipelineExecutionSummaries[0].sourceRevisions[0].revisionSummary')
- echo COMMIT_MESSAGE $COMMIT_MESSAGE # debug command only

Ref - https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-filter.html

Udit Agrawal
  • 69
  • 7
  • When using this approach with a GitHub Enterprise connection I ran into the issue that the 'revisionSummary' was actually a piece of json: `"{\"ProviderType\":\"GitHubEnterpriseServer\",\"CommitMessage\":\"\"}"`. So I still needed jq to get the message – Fried Hoeben Mar 16 '23 at 09:56