How to Add / Replace array elements with "yq" conditionally

Question

I am looking for some help with using "yq" to replace / add existing array elements by name. Also conditionally? Appreciate your help in this regard.

I have tried the below so far.. but this just merges and I am getting duplicates when I rerun this after updates.

yq eval-all '... comments="" | . as $item ireduce ({}; . *+ $item)' kong-oauth.yaml kong-apikey.yaml

Following scenario, I like to updated "Master.yaml" 'services' array only by name.

Master.yaml

services:
 - name: service1
   url: https://service1.microservice.net
 - name: service2
   url: https://service2.microservice.net

service3.yaml

services:
 - name: service3
   url: https://service3.microservice.net

service2.yaml

services:
 - name: service2
   url: https://service22.microservice.net

Expected YAML

services:
 - name: service1
   url: https://service1.microservice.net
 - name: service2
   url: https://service22.microservice.net
 - name: service3
   url: https://service3.microservice.net

Thanks Sanjay

mike.f · Accepted Answer · 2022-08-18T00:06:45.063

8

I don't have enough rep to comment on your answer @kev - but that is super neat. I'm going to clean it up a little and put it into the tips and tricks in the yq docs.

I've made it a little more generic and put it all in one command:

yq ea '.services = (
  ((.services[] | {.name: .}) as $item ireduce ({}; . * $item )) as $uniqueMap
  | ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
) | select(fi==0)' *.yaml

Disclosure: I wrote yq

edited Aug 18 '22 at 00:06

answered Nov 29 '21 at 09:35

mike.f

1,586
13
14

Cool! `yq` really is a swiss-army knife for YAML! – kev Nov 30 '21 at 03:13
What's needed to preserve other fields (eg at the same level as `.services`)? – Noel Yap Aug 16 '22 at 22:31
1

Good question @NoelYap - I've updated the example to do just that. Basically I wrapped the original expression and assigned it to '.services', and then selected the first files yaml – mike.f Aug 18 '22 at 00:06
Awesome! What would make the answer awesomer would be a section breakdown of the solution with explanations. For example, "$uniqueMap is assigned … then it's …; .services is set to that output then that's piped to select(fi==0) to …". I'm specifically interested in what `select(fi==0)` does exactly especially since the docs don't have any examples similar to that. – Noel Yap Aug 19 '22 at 17:11
Just found `select(fi == 0)` in https://mikefarah.gitbook.io/yq/operators/file-operators. Still, a breakdown with explanations and even links to the relevant operator docs would make the answer much more helpful. – Noel Yap Aug 19 '22 at 17:13

score 4 · Answer 2 · answered Nov 25 '21 at 10:32

ireduce({}; . * $item) - create an object
ireduce([]; . + $item) - create an array

yq ea '(.services[] | {.name: .url}) as $item ireduce ({}; . * $item ) | to_entries' Master.yaml service*.yaml |
  yq e '{"services": .[] as $item ireduce([]; . + {"name": $item.key, "url": $item.value})}' -

How to Add / Replace array elements with "yq" conditionally

2 Answers2

Linked