Ansible how to select hosts based on certain attributes and use their ip addresses to create a list at runtime

Question

I have a specific question about data manipulation in ansible.

In my inventory file, I have a group called postgresql as below:

[postgresql]
host1 ansible_host=1.1.1.1 postgresql_cluster_port=5432 postgresql_harole=master
host2 ansible_host=2.2.2.2 postgresql_cluster_port=5432 postgresql_harole=slave postgresql_master_ip=1.1.1.1
host3 ansible_host=3.3.3.3 postgresql_cluster_port=5432 postgresql_harole=slave postgresql_master_ip=1.1.1.1
host4 ansible_host=4.4.4.4 postgresql_cluster_port=5432 postgresql_harole=slave postgresql_master_ip=1.1.1.1

Somewhere in my playbook I will need to manipulate and use filters to make a list of ip addresses of all hosts whose postgresql_harole=slave as below:

- hosts: postgresql
  gather_facts: True
  remote_user: root
  tasks:
    - set_facts:
        slave_ip_list: "{{ expressions }}"

I am pulling my hairs to have the correct expressions... any help is highly appreciated!!!!

Answer 1

The easiest solution is probably to use Ansible's group_by module, which allows you to create groups dynamically based on the values of host variables. For example, given this inventory:

[postgresql]
host1 ansible_host=1.1.1.1 postgresql_cluster_port=5432 postgresql_harole=master
host2 ansible_host=2.2.2.2 postgresql_cluster_port=5432 postgresql_harole=worker postgresql_master_ip=1.1.1.1
host3 ansible_host=3.3.3.3 postgresql_cluster_port=5432 postgresql_harole=worker postgresql_master_ip=1.1.1.1
host4 ansible_host=4.4.4.4 postgresql_cluster_port=5432 postgresql_harole=worker postgresql_master_ip=1.1.1.1

We can group hosts using the postgresql_harole variable like this:

- hosts: all
  gather_facts: false
  tasks:
    - name: create group of postgresql workers
      group_by:
        key: "pg_role_{{ postgresql_harole|default('none') }}"

- hosts: pg_role_worker
  gather_facts: false
  tasks:
    - run_once: true
      debug:
        var: groups.pg_role_worker

    - debug:
        msg: "Hello world"

Running the above playbook would generate output similar to:

PLAY [all] ***********************************************************************************

TASK [create group of postgresql workers] ****************************************************
changed: [host1]
changed: [host2]
changed: [host3]
changed: [host4]

PLAY [pg_role_worker] ************************************************************************

TASK [debug] *********************************************************************************
ok: [host2] => {
    "groups.pg_role_worker": [
        "host2",
        "host3",
        "host4"
    ]
}

TASK [debug] *********************************************************************************
ok: [host2] => {
    "msg": "Hello world"
}
ok: [host3] => {
    "msg": "Hello world"
}
ok: [host4] => {
    "msg": "Hello world"
}

PLAY RECAP ***********************************************************************************
host1                      : ok=1    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host2                      : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host3                      : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host4                      : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

You can see that (a) there now exists a group pg_role_worker containing hosts host2, host3, and host4 and that (b) the final debug task runs only on those hosts.

Answer 2

Q: "List of ip addresses of all hosts whose postgresql_harole=slave"

A: Select the attribute from hostvars, e.g.

    - set_fact:
        slave_ip_list: "{{ hostvars|dict2items|
                           selectattr('value.postgresql_harole', 'eq', 'slave')|
                           map(attribute='value.ansible_host')|
                           list }}"
      run_once: true

gives

  slave_ip_list:
  - 2.2.2.2
  - 3.3.3.3
  - 4.4.4.4

---

Select the hostvars for the group postgresql first if there are hostvars for other hosts, e.g. as a result of - hosts: all. The task below gives the same result

    - set_fact:
        slave_ip_list: "{{ groups.postgresql|
                           map('extract', hostvars)|
                           selectattr('postgresql_harole', 'eq', 'slave')|
                           map(attribute='ansible_host')|
                           list }}"
      run_once: true

---

Update

You can simplify both the code and inventory. Put the declarations into the group_vars. For example,

shell> cat group_vars/postgresql
postgresql_cluster_port: 5432
postgresql_master_ip: "{{ groups.postgresql|
                          map('extract', hostvars)|
                          selectattr('postgresql_harole', 'eq', 'master')|
                          map(attribute='ansible_host')|first }}"
postgresql_slave_ip: "{{ groups.postgresql|
                         map('extract', hostvars)|
                         selectattr('postgresql_harole', 'eq', 'slave')|
                         map(attribute='ansible_host')|list }}"

Then, you can remove postgresql_master_ip and postgresql_cluster_port from the inventory

shell> cat hosts
[postgresql]
host1 ansible_host=1.1.1.1 postgresql_harole=master
host2 ansible_host=2.2.2.2 postgresql_harole=slave
host3 ansible_host=3.3.3.3 postgresql_harole=slave
host4 ansible_host=4.4.4.4 postgresql_harole=slave

The playbook

- hosts: postgresql
  gather_facts: false
  tasks:
    - debug:
        msg: |
          ansible_host: {{ ansible_host }}
          postgresql_harole: {{ postgresql_harole }}
          postgresql_master_ip: {{ postgresql_master_ip }}
          postgresql_cluster_port: {{ postgresql_cluster_port }}
          postgresql_slave_ip: {{ postgresql_slave_ip }}

gives

shell> ansible-playbook pb.yml 

PLAY [postgresql] ****************************************************************************

TASK [debug] *********************************************************************************
ok: [host1] => 
  msg: |-
    ansible_host: 1.1.1.1
    postgresql_harole: master
    postgresql_master_ip: 1.1.1.1
    postgresql_cluster_port: 5432
    postgresql_slave_ip: ['2.2.2.2', '3.3.3.3', '4.4.4.4']
ok: [host2] => 
  msg: |-
    ansible_host: 2.2.2.2
    postgresql_harole: slave
    postgresql_master_ip: 1.1.1.1
    postgresql_cluster_port: 5432
    postgresql_slave_ip: ['2.2.2.2', '3.3.3.3', '4.4.4.4']
ok: [host4] => 
  msg: |-
    ansible_host: 4.4.4.4
    postgresql_harole: slave
    postgresql_master_ip: 1.1.1.1
    postgresql_cluster_port: 5432
    postgresql_slave_ip: ['2.2.2.2', '3.3.3.3', '4.4.4.4']
ok: [host3] => 
  msg: |-
    ansible_host: 3.3.3.3
    postgresql_harole: slave
    postgresql_master_ip: 1.1.1.1
    postgresql_cluster_port: 5432
    postgresql_slave_ip: ['2.2.2.2', '3.3.3.3', '4.4.4.4']

PLAY RECAP ***********************************************************************************
host1: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host2: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host3: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
host4: ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0