Extract nested arrays from dict using jmespath

Question

I have this array of dicts containing arrays out of the Ansible output of an acme_certificate for DNS challenges ({{ le_challenges.results | json_query('[].challenge_data_dns') }}):

[
  {
    "_acme-challenge.foo.example.de": [
      "<token1>"
    ],
    "_acme-challenge.bar.example.de": [
      "<token2>",
      "<token3>"
    ]
  },
  {
    "_acme-challenge.baz.example.de": [
      "<token4>"
    ]
  }
]

As I do not need the individual _acme-challenge records because all domains in question are CNAME-aliased towards a single Route53-hosted zone, I need the above output to be transformed to this so that I can run a loop over route53 actions:

[
  "<token1>",
  "<token2>",
  "<token3>",
  "<token4>"
]

I tried all possible kinds of experiments including dict2list and map on the Ansible side, but I cannot find a way to accomplish this transformation since the keys containing the tokens have dynamic names. Any ideas?

=> `{{ le_challenges.results | json_query('[].challenge_data_dns') | combine | dict2items | map(attribute='value') | flatten }}` — Zeitounator, May 20 '22 at 15:49

β.εηοιτ.βε · Accepted Answer · 2022-05-20T20:14:45.057

Querying any key of a dictionary is as simple as using a .* in JMESPath, you will end up with a list of list of token, then.

With that, there is a flatten operator, which is simply [], you need to repeat it for every level of list you want to flatten.

This said, you will have to reset the projection you made first: with a pipe operator — |.

So, your query ends up being:

[].challenge_data_dns.* | [][]

Given the task:

- debug:
    var: >-
      le_challenges.results | json_query(
        '[].challenge_data_dns.* | [][]'
      )
  vars:
    le_challenges:
      results:
        - challenge_data_dns:
            _acme-challenge.foo.example.de:
              - "<token1>"
            _acme-challenge.bar.example.de:
              - "<token2>"
              - "<token3>"
        - challenge_data_dns:
            _acme-challenge.baz.example.de:
              - "<token4>"

This yields:

? |-
  le_challenges.results | json_query(
    '[].challenge_data_dns.* | [][]'
  )
: - <token1>
  - <token2>
  - <token3>
  - <token4>

score 0 · Answer 2 · answered May 21 '22 at 11:38

Generally, in the dictionaries challenge_data_dns, there might be more attributes along with _acme-challenge.*.example.de. Then, the expressions below do the job

    le_tokens: "{{ le_challenges.results|
                   map(attribute='challenge_data_dns')|
                   map('dict2items')|
                   map('selectattr', 'key', 'match', key_match)|
                   map('map', attribute='value')|
                   flatten }}"
    key_match: '_acme-challenge\.(.*)\.example\.de'

gives

  le_tokens:
  - <token1>
  - <token2>
  - <token3>
  - <token4>

^{Notes
Tested with the data below
le_challenges:
results:
- challenge_data_dns:
_acme-challenge.foo.example.de:
- <token1>
b2: test
- challenge_data_dns:
_acme-challenge.bar.example.de:
- <token2>
- <token3>
- challenge_data_dns:
_acme-challenge.baz.example.de:
- <token4>}

Extract nested arrays from dict using jmespath

2 Answers2