1

I'm creating a requirements.yml in an Ansible project, and I want to identify all of the modules that need to be installed from ansible-galaxy that are used by project playbooks. ansible-doc --list --playbook-dir foo seems like the right tool for the job, but it lists all locally available modules, not just the ones which are actually used in the foo directory. ansible-galaxy list doesn't account for any which are needed but not installed.

Is there a way to do this where I don't end up writing a shell script to sed|awk|grep the info I want?

The best approach I've been able to come up with so far is to ansible-playbook --syntax-check each of the playbooks. This will throw errors such as

ERROR! the role 'foo' was not found ...
ERROR! couldn't resolve module/action 'bar'. This often indicates a misspelling, missing collection, or incorrect module path.

but this is not ideal because it exits as soon as any error occurs. I have to fix each one and run the syntax check again.

Florian
  • 2,562
  • 5
  • 25
  • 35
JR.
  • 311
  • 1
  • 13
  • 3
    That problem sounds astronomically hard. Are your playbooks so massive that it's unreasonable to actually audit them? Alternatively, sure, the iterative version is wasteful, but it _will_ eventually do what you want, right? – mdaniel Feb 20 '21 at 06:11
  • 1
    The project playbooks, for the most part, are not excessively long, though the project has quite a lot of playbooks and roles. You are right, I was eventually able to suss out all of the dependencies using the iterative method. I also made sure that the repository will run a `--check-syntax` on all playbooks as part of CI to catch any requirements added in the future. – JR. Feb 21 '21 at 20:01

1 Answers1

3

FWIW, as a concept, the playbook below lists the modules used in a role

- hosts: localhost

  vars:
    keywords:
      - always
      - become
      - block
      - loop
      - loop_control
      - name
      - notify
      - register
      - tags
      - vars
      - when

  tasks:

    - name: The variable my_role_path is mandatory
      assert:
        that: my_role_path|d('')|length > 0

    - name: Find tasks files
      find:
        path: "{{ my_role_path }}/tasks"
        patterns: '*.yml,*.yaml'
        recurse: true
      register: result

    - name: Create list of tasks
      set_fact:
        lft: "{{ lft|d([]) + lookup('file', item)|from_yaml }}"
      loop: "{{ result.files|map(attribute='path')|list }}"

    - name: Get list of keys
      set_fact:
        lfk: "{{ lfk|d([]) + item.keys()|list }}"
      loop: "{{ lft }}"

    - name: Get list of keys from block/rescue/always
      set_fact:
        lfk: "{{ lfk|d([]) + item.keys()|list }}"
      loop: "{{ lft|json_query('[].[block, rescue, always]')|flatten }}"

    - name: Display list of modules
      debug:
        var: lfk|unique|sort|difference(keywords)

For example, analyze the role systemd

shell> ansible-playbook pb.yml -e my_role_path=roles/ansible-role-systemd

...

  lfk|unique|sort|difference(keywords):
  - command
  - file
  - include_tasks
  - meta
  - systemd
  - template

Complete the list of the keywords.

Use the tasks below to analyze a playbook

  tasks:

    - name: The variable my_playbook_path is mandatory
      assert:
        that: my_playbook_path|d('')|length > 0

    - name: Create list of tasks
      set_fact:
        lft: "{{ _playbook|map(attribute='tasks')|flatten }}"
      vars:
        _playbook: "{{ lookup('file', my_playbook_path)|from_yaml }}"

    - name: Get list of keys
      set_fact:
        lfk: "{{ lfk|d([]) + item.keys()|list }}"
      loop: "{{ lft }}"

    - name: Get list of keys from block/rescue/always
      set_fact:
        lfk: "{{ lfk|d([]) + item.keys()|list }}"
      loop: "{{ lft|json_query('[].[block, rescue, always]')|flatten }}"

    - name: Display list of modules
      debug:
        var: lfk|unique|sort|difference(keywords)

For example, analyzing the first playbook gives

  lfk|unique|sort|difference(keywords):
  - assert
  - debug
  - find
  - set_fact
Vladimir Botka
  • 58,131
  • 4
  • 32
  • 63