How to loop over this dictionary in Ansible?

Question

Say I have this dictionary

war_files:
  server1:
  - file1.war
  - file2.war
  server2:
  - file1.war
  - file2.war
  - file3.war

and for now I just want to loop over each item (key), and then over each item in the key (value). I did this

- name: Loop over the dictionary
  debug: msg="Key={{ item.key }} value={{ item.value }}"
  with_dict: "{{ war_files }}"

And I get this. It is of course correct, but is NOT what I want.

ok: [localhost] => (item={'value': [u'file1.war', u'file2.war'], 'key': u'server1'}) => {
    "item": {
        "key": "server1", 
        "value": [
            "file1.war", 
            "file2.war"
        ]
    }, 
    "msg": "Server=server1, WAR=[u'file1.war', u'file2.war']"
}
ok: [localhost] => (item={'value': [u'file1.war', u'file2.war', u'file3.war'], 'key': u'server2'}) => {
    "item": {
        "key": "server2", 
        "value": [
            "file1.war", 
            "file2.war", 
            "file3.war"
        ]
    }, 
    "msg": "Server=server2, WAR=[u'file1.war', u'file2.war', u'file3.war']"
}

I want to get an output that says

"msg": "Server=server1, WAR=file1.war"
"msg": "Server=server1, WAR=file2.war"
"msg": "Server=server2, WAR=file1.war"
"msg": "Server=server2, WAR=file2.war"
"msg": "Server=server2, WAR=file3.war"

IOW, how can I write a task to iterates over the dictionary so it goes through each key, and then the items within each key? In essence, I have a nested array and want to iterate over it?

score 48 · Answer 1 · answered Jun 06 '18 at 03:14

48

Hows this

- hosts: localhost
  vars:
    war_files:
      server1:
      - file1.war
      - file2.war
      server2:
      - file1.war
      - file2.war
      - file3.war
  tasks:
    - name: Loop over subelements of the dictionary
      debug:
        msg: "Key={{ item.0.key }} value={{ item.1 }}"
      loop: "{{ war_files | dict2items | subelements('value') }}"

dict2items, subelements filters are coming in Ansible 2.6.

FYI, if a filter for your objective doesn't exist, you can write your own in python without having to resort to jinja2 hacks. Ansible is easily extendable; filters in filter_plugins/*.py are searched by default adjacent to your plays/roles and are automatically included - see Developing Plugins for details.

answered Jun 06 '18 at 03:14

tmoschou

977
1
8
11

8

Much better. You don't have to use `subelements` in all cases. Try `loop: "{{ users | dict2items }}"` and then `msg: "Key={{ item.key }} value={{ item.value }}` to get a sense of what the possibilities are. See https://docs.ansible.com/ansible/devel/user_guide/playbooks_loops.html and note that `when` is evaluated item by item. – JL Peyret Aug 30 '18 at 23:15
1

Using ansible I've been able to write expressions like `{{ Accounts | dict2items | selectattr('key', 'in', SecuredAccounts) | map(attribute='value.Id') | list }}` to flatten, filter and extract the values I needed. – Federico Dec 08 '20 at 21:55

score 29 · Answer 2 · answered Feb 26 '18 at 16:25

29

Now Ansible allows this

- name: add several users
  user:
    name: "{{ item.name }}"
    state: present
    groups: "{{ item.groups }}"
  with_items:
    - { name: 'testuser1', groups: 'wheel' }
    - { name: 'testuser2', groups: 'root' }

answered Feb 26 '18 at 16:25

Artem Bernatskyi

4,185
2
26
35

22

This is looping over a list of dictionaries, not over the keys of a dictionary as the question asked. @sjas so this should not be the accepted answer ;). – Hadrien TOMA May 20 '18 at 16:11
2

May not be accepted answer but I believe this is the better way to go. – David Weber Nov 09 '18 at 09:30
9

This fundamentally does not address the question. There are three levels to the nesting, which is the 'real' problem. Also to save confusion, try using the example variables, to prove your answer addresses it. – courtlandj Nov 13 '18 at 19:26
4

This answer addresses a "list of dictionaries" which is a pretty basic example. The OP is asking about a "dictionary of dictionaries containing lists", which is dramatically different. No, this answer is not the better way to go and no, this should not be the accepted answer. You are fundamentally misunderstanding the original question. – James Mar 28 '19 at 22:11

Nehal J Wani · Accepted Answer · 2019-04-03T03:36:08.277

EDIT: At the time of writing this answer, Ansible 2.6 wasn't out. Please read the answer provided by @tmoschou, as it is much better.

Well, I couldn't find a very easy way to do it, however, with a little bit of jinja2, we can achieve something of this sort:

/tmp ❯❯❯ cat example.yml
---
- hosts: 127.0.0.1
  vars:
    war_files:
      server1:
      - file1.war
      - file2.war
      server2:
      - file1.war
      - file2.war
      - file3.war
  tasks:
  - set_fact:
      war_files_list_of_dicts: |
          {% set res = [] -%}
          {% for key in war_files.keys() -%}
             {% for value in war_files[key] -%}
              {% set ignored = res.extend([{'Server': key, 'WAR':value}]) -%}
             {%- endfor %}
          {%- endfor %}
          {{ res }}

  - name: let's debug the crap out of this
    debug: var=war_files_list_of_dicts

  - name: Servers and their WARs!!!
    debug:
       msg: "Server={{ item.Server }}, WAR={{ item.WAR }}"
    with_items: "{{ war_files_list_of_dicts }}"

And, when the playbook is run:

/tmp ❯❯❯ ansible-playbook example.yml
 [WARNING]: provided hosts list is empty, only localhost is available


PLAY [127.0.0.1] ***************************************************************

TASK [setup] *******************************************************************
ok: [127.0.0.1]

TASK [set_fact] ****************************************************************
ok: [127.0.0.1]

TASK [let's debug the crap out of this] ****************************************
ok: [127.0.0.1] => {
    "war_files_list_of_dicts": [
        {
            "Server": "server1", 
            "WAR": "file1.war"
        }, 
        {
            "Server": "server1", 
            "WAR": "file2.war"
        }, 
        {
            "Server": "server2", 
            "WAR": "file1.war"
        }, 
        {
            "Server": "server2", 
            "WAR": "file2.war"
        }, 
        {
            "Server": "server2", 
            "WAR": "file3.war"
        }
    ]
}

TASK [Servers and their WARs!!!] ***********************************************
ok: [127.0.0.1] => (item={'WAR': u'file1.war', 'Server': u'server1'}) => {
    "item": {
        "Server": "server1", 
        "WAR": "file1.war"
    }, 
    "msg": "Server=server1, WAR=file1.war"
}
ok: [127.0.0.1] => (item={'WAR': u'file2.war', 'Server': u'server1'}) => {
    "item": {
        "Server": "server1", 
        "WAR": "file2.war"
    }, 
    "msg": "Server=server1, WAR=file2.war"
}
ok: [127.0.0.1] => (item={'WAR': u'file1.war', 'Server': u'server2'}) => {
    "item": {
        "Server": "server2", 
        "WAR": "file1.war"
    }, 
    "msg": "Server=server2, WAR=file1.war"
}
ok: [127.0.0.1] => (item={'WAR': u'file2.war', 'Server': u'server2'}) => {
    "item": {
        "Server": "server2", 
        "WAR": "file2.war"
    }, 
    "msg": "Server=server2, WAR=file2.war"
}
ok: [127.0.0.1] => (item={'WAR': u'file3.war', 'Server': u'server2'}) => {
    "item": {
        "Server": "server2", 
        "WAR": "file3.war"
    }, 
    "msg": "Server=server2, WAR=file3.war"
}

PLAY RECAP *********************************************************************
127.0.0.1                  : ok=4    changed=0    unreachable=0    failed=0

Thank you! This is where I like Chef over Ansible, where I can just write Ruby to iterate over a data structure. In essence that's what you did here with Python, but that syntax is just ugly. — Chris F, Feb 14 '17 at 15:18
But yes, I agree with you. Puppet and Chef have an edge over Ansible because of this. — Nehal J Wani, Feb 14 '17 at 16:33
Better use `dict2items` and `subelements` mentioned by tmoschou. This solution also works but increases complexity a lot. — phobie, Feb 12 '19 at 12:24

Cameron Lowell Palmer · Answer 4 · 2018-12-20T09:51:48.927

dict2items

I found myself wanting to iterate over a heterogeneous set of keys and their associated values and use the key-value pair in a task. The dict2items filter is the least painful way I've found. You can find dict2items in Ansible 2.6

Example Dict

systemsetup:
  remotelogin: "On"
  timezone: "Europe/Oslo"
  usingnetworktime: "On"
  sleep: 0
  computersleep: 0
  displaysleep: 0
  harddisksleep: 0
  allowpowerbuttontosleepcomputer: "Off"
  wakeonnetworkaccess: "On"
  restartfreeze: "On"
  restartpowerfailure: "On"

Example Task

---
- debug:
    msg: "KEY: {{ item.key }}, VALUE: {{ item.value }}"
  loop: "{{ systemsetup | dict2items }}"

M_dk · Answer 5 · 2020-02-10T07:31:39.377

Here is my preferred way to loop over dictionaries:

input_data.yml contains the following:

----
input_data:
  item_1:
    id: 1
    info: "Info field number 1"
  item_2:
    id: 2
    info: "Info field number 2"

I then use a data structure like the above in a play using the keys() function and iterate over the data using with_items:

---
- hosts: localhost
  gather_facts: false
  connection: local
  tasks:
    - name: Include dictionary data
      include_vars:
        file: data.yml

    - name: Show info field from data.yml
      debug:
        msg: "Id: {{ input_data[item]['id'] }} - info: {{ input_data[item]['info'] }}"
      with_items: "{{ input_data.keys() | list }}"

The above playbook produces the following output:

PLAY [localhost] ***********************************************************

TASK [Include dictionary data] *********************************************
ok: [localhost]

TASK [Show info field from data.yml] ***************************************
ok: [localhost] => (item=item_2) => {
    "msg": "Id: 2 - info: Info field item 2"
}
ok: [localhost] => (item=item_3) => {
    "msg": "Id: 3 - info: Info field item 3"
}
ok: [localhost] => (item=item_1) => {
    "msg": "Id: 1 - info: Info field item 1"
}

PLAY RECAP *****************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0

Can you explain how this addresses the original question? Show how you define one of the item_ entries with more than one id & value. — courtlandj, Nov 13 '18 at 19:30
My answer directly addresses the title of the question so my answer is helpful for people looking for an answer to that. — M_dk, Nov 14 '18 at 09:52
I prefer this to dict2items in 9 out of 10 situations. You can just convert dict|list right away skipping keys(), this will give list of top level dict keys as well. — Vsevolod, Jul 29 '21 at 12:16

The Fool · Answer 6 · 2021-03-30T22:37:22.300

3

One way of doing it that worked for me was using with_dict. Note the dict should not be named. Just the key value pairs.

- name: ssh config
  lineinfile:
    dest: /etc/ssh/sshd_config
    regexp: '^#?\s*{{item.key}}\s'
    line: '{{item.key}} {{item.value}}'
    state: present
  with_dict: 
    LoginGraceTime: "1m"
    PermitRootLogin: "yes"
    PubkeyAuthentication: "yes"
    PasswordAuthentication: "no"
    PermitEmptyPasswords: "no"
    IgnoreRhosts: "yes"
    Protocol: 2

edited Mar 30 '21 at 22:37

answered Mar 30 '21 at 22:18

The Fool

16,715
5
52
86

The `dict2items` solutions are much better because they can work with both `with_items` and `with_nested`. – bfontaine Sep 15 '22 at 16:21
1

@bfontaine, for my given example, nothing is better, if the outcome is the same. If anything, using `dict2items` is overkill, for my use case. It's Unnecessary complexity, aka overengineering. `with_dict` exists for a reason. Namely, when it's sufficient to use it, so you don't need extra function calls and loops. In other cases it may be not sufficinet. So as always, it depends. – The Fool Sep 15 '22 at 19:32
Let me start with, I love this answer. +1. But Overkill? Overengineering? `dict2items` works and the expression is compact. IIRC I did it with dict2items because I have settings separate in a very large bootstrap project. – Cameron Lowell Palmer May 05 '23 at 07:10

How to loop over this dictionary in Ansible?

6 Answers6

dict2items

Example Dict

Example Task

Linked