Substitute Yaml values with yq by new value as output of bash function which accept current value as input

Question

Overview

• Something like Substitution with sed + bash function But using yq
• yq of kislyuk not other yq projects https://github.com/kislyuk/yq (pip install yq)

Details

I have this bash function :

function replace_image_repo() {
  echo "nexus.local/${1}"
}

On the other side, i have a YAML file:

# pod.yaml
kind: Pod
# ...
spec:
  containers:
    - name: web
      image: nginx
    - name: logger
      image: abdennour/logger

I am able to replace all value occurences of .image key by a static value:

yq -y '(.. | .image?) |= "mynewimage"' pod.yaml

And the result is as expected:

# pod.yaml
kind: Pod
# ...
spec:
  containers:
    - name: web
      image: mynewimage # <-- Replacement done successfully
    - name: logger
      image: mynewimage # <-- Replacement done successfully

However, I want to leverage the bash function above replace_image_repo and call it to calculate the new value for each occurrence based on the current value :

• for example, nginx must be replaced by the output of $(replace_image_repo nginx) which should be nexus.local/nginx.

• Is it possible to match the current value ?

• If so , is it possible a call the Bash function "yq -y '.... $(HERE)'" ?

Answer 1

You can do much better than that. Since https://github.com/kislyuk/yq makes use of the prowess of jq underneath, you can use the latter's --arg fields to pass in the values that you want to replace. For e.g. your case could be customized to pass the old and the new replacement strings to be passed.

Also the jq filter expression (.. | .image?) |= "mynewimage" is not the best of approaches, as .. uses a recursive descent parsing, you might end up null values in your modified result. The right approach would be to modify the filter to match the exact object containing the string and replace with the target value.

Recommend dropping the non-standard keyword function from shell functions esp. bash

replace_image_repo() {
  printf '%s' "nexus.local/${1}"
}

and use yq as

yq -y --arg old "nginx" \
      --arg new "$(replace_image_repo "nginx")" \
      '.spec.containers |= map( select(.image == $old).image = $new )' yaml

Or if your requirement is to apply the substitution to all .image fields under containers, you can do below without using a shell function.

yq -y '.spec.containers |= map(.image = "nexus.local/\(.image)")' yaml 

You can customize it further by passing the prefix string as an argument

yq -y --arg prefix "nexus.local/" '.spec.containers |= map(.image = ($prefix + "\(.image)") )' yaml 

---

Considering your argument about having to use a much complex shell function, you can do the following approach. The two parse on the YAML first to get the new image names based on the shell function (complex, now abstracted) and then later re-use the results to replace the image names on the original file back.

This is because jq doesn't allow executing arbitrary shell functions in its expression context yet.

#!/usr/bin/env bash

replace_image_repo() {
  printf '%s' "nexus.local/${1}"
}


convert_to_array() {
    local results=()
    while IFS= read -r image; do
        results+=( \"$(replace_image_repo "$image")\" )
    done < <(yq -r '.spec.containers[].image' yaml)
    local IFS=","
    printf '[%s]' "${results[*]}"
}


yq -y --argjson images "$(convert_to_array)" \
    'reduce range(0; $images | length) as $i (.;
       .spec.containers[$i].image = $images[$i])' yaml