FileNotFoundError: [Errno 2] No such file or directory : Ansible

Question

I'm new to Ansible, here I created a module which takes CSV as an argument like below: But it keep throwing error that it's not able to find CSV_TEST.csv file. am I creating it the right way as a variable in the ansible module?

module.py

def main():
    field = dict(
        csv=dict(type='str', required=True)
    )

    result = dict(
        changed=False,
        response='')

    module = AnsibleModule(argument_spec=field)

    csvFile = module.params['csv']

    listFinal = list()
    final = list()

    if csvFile:
        with open(csvFile, "r", newline='', encoding='utf-8-sig') as csvImport:
            reader = csv.DictReader(csvImport)
            for row in reader:
                newDict = dict({
                    'name': row.get("Name"),
                    'address': row.get("Address")
                    'val': row.get("Val")
                })
                listFinal.append(newDict)


    if listFinal:
        for entry in listFinal:
            if entry.get("val") == "Incorrect":
                name = entry.get('name')
                add = entry.get('address')
                update = f'set add {add} of {name}'

                final.append(update)

        result['final'] = final

    module.exit_json(changed=False, meta=result)


if __name__ == '__main__':
    main()

Playbook.yml

---
- name: Test Variables with Ansible
  hosts: localhost
  vars:
    x: 30
    xName: "Sai"
  gather_facts: false
  become: false
  tasks:
    - name: Test Device Validation
      portDescription:
        csv: CSV_TEST.csv
      register: result

    - debug: var=result

The error is:

***
FileNotFoundError: [Errno 2] No such file or directory: 'CSV_TEST.csv'
fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Shared connection to localhost closed.\r\n",
***

Can someone please suggest, what am I doing wrong here? CSV_TEST.csv is under the same tree structure (parent folder) like Playbook.yml

Are you running your ansible-playbook command from the same directory that CSV_TEST.csv is in? — FailureGod, Oct 15 '20 at 21:02
But are you running the command in the same folder? Also one thing you can is log the current directory from the python script with os.getcwd() to see where the python script is executing from. — FailureGod, Oct 15 '20 at 21:13
@FailureGod yea command is being run under same directory. sure let me do that — Shivrai, Oct 15 '20 at 21:16
Jeeezzzz this is tough... okay im grasping at straws here but. If you do log `list(Path(".").iterdir())` (dont forget from pathlib import Path) and CSV_TEST.csv shows up then go ahead and select it from the list and use `read_text()` to get the contents. If that works then I have no idea what's wrong with your current code but at least you'll have a workable alternative. — FailureGod, Oct 15 '20 at 21:37
Please make sure the code you paste is minimal, complete and correct: missing interpreter line (mandatory for ansible module), missing imports (i.e. `AnsibleModule` and `csv` are not defined), syntax error (missing coma in `newDict` definition)... The error you are reporting does not correspond to the code you are showing. Ideally, one should be able to copy/paste your example, run it, and get the exact same error message (delta the version of ansible in use...). Once I fix the above errors, I can't reproduce your issue (but the result is empty). Running ansible 2.10. — Zeitounator, Oct 16 '20 at 07:22
@Zeitounator the imports are understandable that's why I focused on the code portion. Also, I don't see any missing comma in newDict, could you share the code section? Also think what can be the issue before downlinking the question — Shivrai, Oct 16 '20 at 17:08
Please read the [help section](/help) to understand how the site works. Although your imports are understandable, they might be the clue to a problem. Besides that, just follow the site rule: give a minimal, complete and reproducible example of the code. The missing coma is at that line `'address': row.get("Address")`. You can simply copy/paste the code in you question above to get a syntax error from your module. — Zeitounator, Oct 16 '20 at 17:14
@FailureGod : As you said I tried the command : list(Path("Ansible/CSV_TEST.csv").read_text()) because my entire project runs under Ansible subdirectory. And CSV_TEST.csv shows all the content. ParentDiretory -> (Ansible, Venv) 2 subdirectories — Shivrai, Oct 16 '20 at 17:36
@Zeitounator they would have to post a docker container for anyone to be able to replicate this. Their environment variables, file path where they put the custom module, and full directory structure, among other things are all important. It would be nice to have a docker container but it would be ridiculous to punish someone because they don't produce it. Also the fact that `csvFile` isn't modified points to a problem with the overall setup and not the code itself. — FailureGod, Oct 16 '20 at 19:26
@FailureGod I don't get your comment. I'm only asking for a correct module file that anyone can copy paste to test right away, an [MCVE](/help/mcve) as described in the help. Moreover, I don't see anything in this code that is suppose to modify the csv file. And last: I reproduce the issue (after fixing the example) only if there is no csv file. — Zeitounator, Oct 17 '20 at 07:39
@FailureGod See my below answer for an MCVE. As you will see, there is absolutely no need of a docker image for that. — Zeitounator, Oct 17 '20 at 11:09

Zeitounator · Answer 1 · 2020-10-17T11:10:24.033

Once I fix the problems I reported in my above comment, I don't see anything else wrong with your code. There might be logic problems (but I have no idea what is the exact expectation). For now, here is a proof run that it works as expected from what I can read in your module code. The paths below are all relative to the current working dir where I launched the playbook from.

The module stored in ./library/port_description.py

#!/usr/bin/python
from ansible.module_utils.basic import *
import csv

def main():
    field = dict(
        csv=dict(type='str', required=True)
    )

    result = dict(
        changed=False,
        response='')

    module = AnsibleModule(argument_spec=field)

    csvFile = module.params['csv']

    listFinal = list()
    final = list()

    if csvFile:
        with open(csvFile, "r", newline='', encoding='utf-8-sig') as csvImport:
            module.debug("csv openned")
            reader = csv.DictReader(csvImport)
            for row in reader:
                newDict = dict({
                    'name': row.get("Name"),
                    'address': row.get("Address"),
                    'val': row.get("Val")
                })
                listFinal.append(newDict)

    if listFinal:
        for entry in listFinal:
            if entry.get("val") == "Incorrect":
                name = entry.get('name')
                add = entry.get('address')
                update = f'set add {add} of {name}'

                final.append(update)

        result['final'] = final

    module.exit_json(changed=False, meta=result)


if __name__ == '__main__':
    main()

The csv file stored in ./CSV_TEST.csv (tried to create something aligned to expected names and python code...)

Name,Address,Val
toto,127.0.0.1,3
titi,192.168.0.1,Incorrect

The playbook stored in ./test.yml

---
- name: Use a custom module reading csv file
  hosts: localhost
  gather_facts: false

  tasks:
    - name: Use our module and register result
      port_description:
        csv: CSV_TEST.csv
      register: port_result

    - name: Show the result
      debug:
        var: port_result

And the result:

$ ansible-playbook test.yml 

PLAY [Use a custom module reading csv file] ********************************************************************************************************************************************************************************************

TASK [Use our module and register result] **********************************************************************************************************************************************************************************************
ok: [localhost]

TASK [Show the result] *****************************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "port_result": {
        "changed": false,
        "failed": false,
        "meta": {
            "changed": false,
            "final": [
                "set add 192.168.0.1 of titi"
            ],
            "response": ""
        }
    }
}

PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Note that I can easilly reproduce your exact problem if I remove the given CSV file from its expected location:

$ mv CSV_TEST.csv CSV_TEST.csv.BAK
$ ansible-playbook test.yml 

PLAY [Use a custom module reading csv file] ********************************************************************************************************************************************************************************************

TASK [Use our module and register result] **********************************************************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: FileNotFoundError: [Errno 2] No such file or directory: 'CSV_TEST.csv'
fatal: [localhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/home/user/.ansible/tmp/ansible-tmp-1602932173.7323837-24457-268203592954263/AnsiballZ_port_description.py\", line 102, in <module>\n    _ansiballz_main()\n  File \"/home/user/.ansible/tmp/ansible-tmp-1602932173.7323837-24457-268203592954263/AnsiballZ_port_description.py\", line 94, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/home/user/.ansible/tmp/ansible-tmp-1602932173.7323837-24457-268203592954263/AnsiballZ_port_description.py\", line 40, in invoke_module\n    runpy.run_module(mod_name='ansible.modules.port_description', init_globals=None, run_name='__main__', alter_sys=True)\n  File \"/usr/lib/python3.6/runpy.py\", line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib/python3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_port_description_payload_6v82vnu_/ansible_port_description_payload.zip/ansible/modules/port_description.py\", line 48, in <module>\n  File \"/tmp/ansible_port_description_payload_6v82vnu_/ansible_port_description_payload.zip/ansible/modules/port_description.py\", line 22, in main\nFileNotFoundError: [Errno 2] No such file or directory: 'CSV_TEST.csv'\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

it's the exact same thing I am doing, still no luck. I have kept both .csv and .yml files under same directory and the code looks simple and straight. Don't know why it's not picking it up. — Shivrai, Oct 20 '20 at 15:48

score 0 · Answer 2 · answered Oct 16 '20 at 19:17

I think that if you have a relative file path in Ansible, Ansible will look in the local directory first (that is whatever directory is output if you run ran ansible from) then it looks in some other locations (check out the role directory structure for info on that).

So in my mind your directory structure is like

WorkingDir
|- playbook.yml
|- Ansible
   |- CSV_TEST.csv

And I think you are running ansible like ansible-playbook playbook.yml from inside WorkingDir. It can't find CSV_TEST.csv because it's looking in WorkingDir.

If inside the python you have to put Ansible/CSV_TEST.csv then you also have to put Ansible/CSV_TEST.csv in your playbook. So

---
- name: Test Variables with Ansible
  hosts: localhost
  vars:
    x: 30
    xName: "Sai"
  gather_facts: false
  become: false
  tasks:
    - name: Test Device Validation
      portDescription:
        csv: Ansible/CSV_TEST.csv
      register: result

    - debug: var=result

If this doesn't help then can you write out the directory structure in a text format and show where everything is executed from and where python is running (what os.getcwd() outputs)?

as it does not take the coding format, so PARENT DIRECTORY = WorkingDir then SUB DIRECTORY = Ansible and every playbook and csv file comes under Ansible sub directory. WorkingDir |- Ansible |- CSV_TEST.csv & playbook.yml ` Output of os.getcwd() '/Users/shivanibali/PycharmProjects/WorkingDir' ` — Shivrai, Oct 16 '20 at 20:22
I'm not sure I understood that last message. You have `WorkingDir/Ansible`, `WorkingDir/CSV_TEST.csv`, and `WorkingDir/playbook.yml`? — FailureGod, Oct 16 '20 at 21:31
sorry let me put that clearly. I have only one main directory, let's say it's : WorkingDir and sub directory is Ansible. The files appear as : WorkingDir/Ansible/CSV_TEST.csv and WorkingDir/Ansible/playbook.yml — Shivrai, Oct 20 '20 at 14:14

FileNotFoundError: [Errno 2] No such file or directory : Ansible

2 Answers2