Is it possible to use abstraction in a YAML file?

Question

Is it possible to use arguments or abstraction to minimize repeating code in a YAML file?

I'm writing a YAML file that triggers a deployment and before and after the deployment I would like to make calls to a slack channel indicating the deployment is starting, and finishing, and also if it fails.

Here is what I've written but it feels too verbose:

example_deploy:
    - call: notify
    in:
      msgText: "Deployment starting for environment *${environment}*"
    - try:
      - ${oneops.environmentCommitAndDeploy(environment = 'production', platform = '${platform}', deployAllPlatforms = false )}
      error:
    - log: "Error trying to deploy: ${lastError.cause}"
    - call: notify
      in:
        msgText: " :fire: Deployment failed for environment *${environment}* http://concord.com/#/process/${txId}/log"
    - exit
  - call: notify
    in:
      msgText: " :party: Deployment succeeded for environment *${environment}* http://concord.com/#/process/${txId}/log"

  notify:
  - task: slack
    in:
      channelId: ${alerts}
      username: ${slackname}
      iconEmoji: ${slackEmojiLooper}
      text: "${msgText}"

Now if I want to have example_deploy_2 and do the same type of thing, do I have to rewrite all that code? or is there a way to have a "function" or abstract the repeated parts of the YAML?

UPDATE I've used call to abstract the calls to slack, but now I'm wondering if I can have a generic call to slack and dynamically update the message - because now I'm repeating the params I'm passing to the blocks of code I've defined to be called

Example

  example_deploy:
    - call: slack_start_deploy
    - try:
      - ${transitionVariableUpdate(platform = '${platform}', environment = '${environment}', component = '${component_ear}' variables = { appVersion = '${BRANCH_NAME}-${BUILD_NUMBER}' })}
      - ${environmentCommitAndDeploy(environment = 'qa', platform = '${platform}', deployAllPlatforms = false )}
      error:
    - log: "Error trying to deploy: ${lastError.cause}"
    - call: slack_deploy_error
    - exit
  - call: slack_deploy_success

  slack_start_deploy:
  - slack.postMessage:
      text: "${entryPoint} Deployment starting for environment *${environment}*"
      channelId: ${alerts}
      username: ${slackname}
      iconEmoji: ${slackEmojiConcord}

  slack_deploy_error:
  - slack.postMessage:
      text: " :fire: ${entryPoint} Deployment failed for environment *${environment}* http://concord.com/#/process/${txId}/log"
      channelId: ${alerts}
      username: ${slackname}
      iconEmoji: ${slackEmojiConcord}

  slack_deploy_success:
  - slack.postMessage:
      text: " :party: Deployment succeeded for environment *${environment}* http://concord.com/#/process/${txId}/log"
      channelId: ${alerts}
      username: ${slackname}
      iconEmoji: ${slackEmojiConcord}

Your example is not valid YAML `error:` is a mapping key that is indented as much as the preceding sequence item, and `- call:` is a sequence item indented as much as the `exampel_deplay:` key. You cannot mix collections like that in YAML — Anthon, Apr 25 '19 at 05:54
@anthon spare me the details - I'm interested in the bigger picture. — zero_cool, Apr 25 '19 at 15:44

score 1 · Accepted Answer · answered Apr 25 '19 at 05:52

The only mechanism in the YAML specification that allows for minimising repetition is using an anchor on a node and referring to that node using an alias. This works for both leaf-nodes (i.e. scalar values of any kind) and for the collection nodes (mappings, sequences). Aliases for anchored collections essentially "replace" the whole subtree underneath the collection.

In addition to that there is the merge key << in a mapping which is implemented by most YAML loaders (usually in the construction phase), where you can have one or more mappings provide key-value pairs for keys that are not specified in the mapping that has the merge key (either directly or through earlier processed merges).

On top of that any program using a YAML loader can extend the loader (usually its construction mechanism, but this could be done earlier during the loading process) as they see fit, but such mechanisms are not considered part of YAML.

The merge mechanism can be deployed on your YAML to reduce the number of lines. If your example is changed to example.yaml:

example_deploy:
  - call: slack_start_deploy
  - try:
    - ${transitionVariableUpdate(platform = '${platform}', environment = '${environment}', component = '${component_ear}' variables = { appVersion = '${BRANCH_NAME}-${BUILD_NUMBER}' })}
    - ${environmentCommitAndDeploy(environment = 'qa', platform = '${platform}', deployAllPlatforms = false )}
error:
  - log: "Error trying to deploy: ${lastError.cause}"
  - call: slack_deploy_error
  - exit
  - call: slack_deploy_success

slack_start_deploy:
- slack.postMessage: &pm
    text: "${entryPoint} Deployment starting for environment *${environment}*"
    channelId: ${alerts}
    username: ${slackname}
    iconEmoji: ${slackEmojiConcord}

slack_deploy_error:
- slack.postMessage:
    text: " :fire: ${entryPoint} Deployment failed for environment *${environment}* http://concord.com/#/process/${txId}/log"
    <<: *pm

slack_deploy_success:
- slack.postMessage:
    text: " :party: Deployment succeeded for environment *${environment}* http://concord.com/#/process/${txId}/log"
    <<: *pm

(Please note that I changed the indentation of your error: and - call: ... lines, as presented your file was invalid YAML)

In the above, the &pm is the anchor for the mapping node with four keys. The *pms are the aliases using this mapping, each time using the original value for text.

The following Python program shows by loading, then dumping how the merge keys are expanded to your original during loading.

import sys
from pathlib import Path
import ruamel.yaml

example = Path('example.yaml')

yaml = ruamel.yaml.YAML(typ='safe')
yaml.default_flow_style = False
data = yaml.load(example)
yaml.dump(data, sys.stdout)

which gives:

error:
- log: 'Error trying to deploy: ${lastError.cause}'
- call: slack_deploy_error
- exit
- call: slack_deploy_success
example_deploy:
- call: slack_start_deploy
- try:
  - ${transitionVariableUpdate(platform = '${platform}', environment = '${environment}',
    component = '${component_ear}' variables = { appVersion = '${BRANCH_NAME}-${BUILD_NUMBER}'
    })}
  - ${environmentCommitAndDeploy(environment = 'qa', platform = '${platform}', deployAllPlatforms
    = false )}
slack_deploy_error:
- slack.postMessage:
    channelId: ${alerts}
    iconEmoji: ${slackEmojiConcord}
    text: ' :fire: ${entryPoint} Deployment failed for environment *${environment}*
      http://concord.com/#/process/${txId}/log'
    username: ${slackname}
slack_deploy_success:
- slack.postMessage:
    channelId: ${alerts}
    iconEmoji: ${slackEmojiConcord}
    text: ' :party: Deployment succeeded for environment *${environment}* http://concord.com/#/process/${txId}/log'
    username: ${slackname}
slack_start_deploy:
- slack.postMessage:
    channelId: ${alerts}
    iconEmoji: ${slackEmojiConcord}
    text: ${entryPoint} Deployment starting for environment *${environment}*
    username: ${slackname}

Is it possible to use abstraction in a YAML file?

1 Answers1