Controlling Ansible playbook execution

Question

Is it possible to control batch execution in percentage, but applying it individually to groups of hosts in the Ansible playbook?

my hosts file:

group_1 - has 5 hosts
group_2 - has 18 hosts
group_3 - has 4 hosts

And my playbook like this:

- name: "Apply percent per group"
  hosts: all
  serial: "30%"

Currently, when I run like this, Ansible concatenate all groups and applies the 30% serial, so it runs 8 hosts per round and linearly, so it ends up getting all the hosts in group_1 at once.

Is there a way to apply this 30% per group individually?

So per batch it would run 1 host from group_1, 5 hosts from group_2 and 1 host from group_3.

It would even give me the possibility to increase the percentage that would still not cause unavailability and decrease execution time.

Vladimir Botka · Accepted Answer · 2023-01-06T20:15:51.640

Create the batch on your own. For example, given the inventory

shell> cat hosts
[group_1]
host_1_[01:05]
[group_2]
host_2_[01:18]
[group_3]
host_3_[01:04]

and the variable my_serial_pc

  my_serial_pc: 30

Declare the variables

  count_1: "{{ groups.group_1|length * my_serial_pc|int // 100 }}"
  count_2: "{{ groups.group_2|length * my_serial_pc|int // 100 }}"
  count_3: "{{ groups.group_3|length * my_serial_pc|int // 100 }}"
  batch_1: "{{ groups.group_1|batch(count_1|int) }}"
  batch_2: "{{ groups.group_2|batch(count_2|int) }}"
  batch_3: "{{ groups.group_3|batch(count_3|int) }}"
  my_serial: "{{ count_1|int + count_2|int + count_3|int }}"

give

  count_1: 1
  count_2: 5
  count_3: 1
  batch_1:
    - [host_1_01]
    - [host_1_02]
    - [host_1_03]
    - [host_1_04]
    - [host_1_05]
  
  batch_2:
    - [host_2_01, host_2_02, host_2_03, host_2_04, host_2_05]
    - [host_2_06, host_2_07, host_2_08, host_2_09, host_2_10]
    - [host_2_11, host_2_12, host_2_13, host_2_14, host_2_15]
    - [host_2_16, host_2_17, host_2_18]
 
  batch_3:
    - [host_3_01]
    - [host_3_02]
    - [host_3_03]
    - [host_3_04]

  my_serial: 7

zip and flatten the batches

  my_hosts: "{{ batch_1|
                zip_longest(batch_2)|map('flatten')|
                zip_longest(batch_3)|map('flatten')|flatten }}"

Create a new group my_group

        - add_host:
            name: "{{ item }}"
            groups: my_group
            my_serial: "{{ my_serial }}"
          loop: "{{ my_hosts }}"

Run the next play with the group my_group and serial my_serial

- hosts: my_group
  serial: "{{ hostvars[groups.my_group.0].my_serial }}"
  tasks:
    - debug:
        var: ansible_play_batch|to_yaml
      run_once: true

gives

^{PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_01] =>
ansible_play_batch|to_yaml: |-
[host_1_01, host_2_01, host_2_02, host_2_03, host_2_04, host_2_05, host_3_01]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_02] =>
ansible_play_batch|to_yaml: |-
[host_1_02, host_2_06, host_2_07, host_2_08, host_2_09, host_2_10, host_3_02]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_03] =>
ansible_play_batch|to_yaml: |-
[host_1_03, host_2_11, host_2_12, host_2_13, host_2_14, host_2_15, host_3_03]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_04] =>
ansible_play_batch|to_yaml: |-
[host_1_04, host_2_16, host_2_17, host_2_18, host_3_04, host_1_05]

Example of a complete playbook for testing
shell> cat pb.yml
- hosts: all

vars:

my_serial_pc: 30
count_1: "{{ groups.group_1|length * my_serial_pc|int // 100 }}"
count_2: "{{ groups.group_2|length * my_serial_pc|int // 100 }}"
count_3: "{{ groups.group_3|length * my_serial_pc|int // 100 }}"
batch_1: "{{ groups.group_1|batch(count_1|int) }}"
batch_2: "{{ groups.group_2|batch(count_2|int) }}"
batch_3: "{{ groups.group_3|batch(count_3|int) }}"
my_hosts: "{{ batch_1|
zip_longest(batch_2)|map('flatten')|
zip_longest(batch_3)|map('flatten')|flatten }}"
my_serial: "{{ count_1|int + count_2|int + count_3|int }}"

tasks:

- block:
- debug:
msg: |
count_1: {{ count_1 }}
count_2: {{ count_2 }}
count_3: {{ count_3 }}
batch_1:
{{ batch_1|to_yaml|indent(2) }}
batch_2:
{{ batch_2|to_yaml|indent(2) }}
batch_3:
{{ batch_3|to_yaml|indent(2) }}
my_serial: {{ my_serial }}
when: debug|d(false)|bool
- add_host:
name: "{{ item }}"
groups: my_group
my_serial: "{{ my_serial }}"
loop: "{{ my_hosts }}"
run_once: true

- hosts: my_group
serial: "{{ hostvars[groups.my_group.0].my_serial }}"
tasks:
- debug:
var: ansible_play_batch|to_yaml
run_once: true

gives
shell> ansible-playbook pb.yml -e debug=true

PLAY [all] ************************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_01] =>
msg: |-
count_1: 1
count_2: 5
count_3: 1
batch_1:
- [host_1_01]
- [host_1_02]
- [host_1_03]
- [host_1_04]
- [host_1_05]

batch_2:
- [host_2_01, host_2_02, host_2_03, host_2_04, host_2_05]
- [host_2_06, host_2_07, host_2_08, host_2_09, host_2_10]
- [host_2_11, host_2_12, host_2_13, host_2_14, host_2_15]
- [host_2_16, host_2_17, host_2_18]

batch_3:
- [host_3_01]
- [host_3_02]
- [host_3_03]
- [host_3_04]

my_serial: 7

TASK [add_host] *******************************************************************************************
changed: [host_1_01] => (item=host_1_01)
changed: [host_1_01] => (item=host_2_01)
changed: [host_1_01] => (item=host_2_02)
changed: [host_1_01] => (item=host_2_03)
changed: [host_1_01] => (item=host_2_04)
changed: [host_1_01] => (item=host_2_05)
changed: [host_1_01] => (item=host_3_01)
changed: [host_1_01] => (item=host_1_02)
changed: [host_1_01] => (item=host_2_06)
changed: [host_1_01] => (item=host_2_07)
changed: [host_1_01] => (item=host_2_08)
changed: [host_1_01] => (item=host_2_09)
changed: [host_1_01] => (item=host_2_10)
changed: [host_1_01] => (item=host_3_02)
changed: [host_1_01] => (item=host_1_03)
changed: [host_1_01] => (item=host_2_11)
changed: [host_1_01] => (item=host_2_12)
changed: [host_1_01] => (item=host_2_13)
changed: [host_1_01] => (item=host_2_14)
changed: [host_1_01] => (item=host_2_15)
changed: [host_1_01] => (item=host_3_03)
changed: [host_1_01] => (item=host_1_04)
changed: [host_1_01] => (item=host_2_16)
changed: [host_1_01] => (item=host_2_17)
changed: [host_1_01] => (item=host_2_18)
changed: [host_1_01] => (item=host_3_04)
changed: [host_1_01] => (item=host_1_05)

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_01] =>
ansible_play_batch|to_yaml: |-
[host_1_01, host_2_01, host_2_02, host_2_03, host_2_04, host_2_05, host_3_01]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_02] =>
ansible_play_batch|to_yaml: |-
[host_1_02, host_2_06, host_2_07, host_2_08, host_2_09, host_2_10, host_3_02]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_03] =>
ansible_play_batch|to_yaml: |-
[host_1_03, host_2_11, host_2_12, host_2_13, host_2_14, host_2_15, host_3_03]

PLAY [my_group] *******************************************************************************************

TASK [debug] **********************************************************************************************
ok: [host_1_04] =>
ansible_play_batch|to_yaml: |-
[host_1_04, host_2_16, host_2_17, host_2_18, host_3_04, host_1_05]

PLAY RECAP ************************************************************************************************
host_1_01: ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host_1_02: ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host_1_03: ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host_1_04: ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0}

U880D · Answer 2 · 2023-01-06T18:14:06.157

0

According Setting the batch size with serial your observation is the expected behavior.

Ansible runs in parallel against all the hosts in the pattern you set in the hosts: field of each play.

So regarding

Is there a way to apply this 30% per group individually?

that means you would need to introducde additionally Patterns: targeting hosts and groups and set a limit to a group.

An other approach might be to implement a rolling update strategy whereby serial would have

... a list that specifies how to slowly increase the number of parallel processes or hosts to execute against. This play will execute on one host, then move to two hosts, then 25% of the hosts ...

See also in examlpe Running a playbook on multiple host groups one at a time.

You could also have more than one hosts section within your playbook, whereby each would start a new play. See i.e. How to specify different hosts for different playbooks in one Ansible script?.

edited Jan 06 '23 at 18:14

answered Jan 06 '23 at 16:22

U880D

8,601
6
24
40

This role has some inventories where I can run in different environments (dev, prd for example) within each inventory I have hosts files. So yes, I can limit passing by parameter at execution. type: `ansible-playbook playname.yml -i inventory --limit group_1` – Maycon Machado Jan 06 '23 at 18:25
However, I have several groups and to apply the individual 30% I would have to run it individually and it takes hours depending on the changes. That's why I'm trying to work on a more performative solution where I can individually apply the percentage within each group but running all groups contained in hosts. It could, for example, be a subscript to iterate between the groups applying the percentage, but I couldn't reach a satisfactory result. – Maycon Machado Jan 06 '23 at 18:27
So isn't than [Running a playbook on multiple host groups one at a time](https://stackoverflow.com/a/65241467/6771046) not what you are looking for? – U880D Jan 06 '23 at 18:37
1

These last 2 links you posted are very interesting, I hadn't seen that possibility yet. let me test a few things here. – Maycon Machado Jan 06 '23 at 18:40

Controlling Ansible playbook execution

2 Answers2

Linked