Ansible - dealing with unexpected STDOUT in custom module

Question

I wrote a custom module that wraps a Python library. I define result dict as follows:

result = dict(
    changed=False,
    original_message='Running DBSCONTROL',
    message=''
)

Later in code I wrap the call to the library into try/except as such:

try:
    dbsc = DbsControl(
        module.params.get('user'),
        module.params.get('host'),
        module.params.get('script')
    )
    dbsc.runme()
    result['message'] = 'DBSCONTROL_SUCCESSFULL'
    module.exit_json(**result)
except Exception as ex:
    result['message'] = str(ex)
    module.fail_json(msg='Failed DBSCONTROL execution', **result)

I do not have a single print statement anywhere in the module and my lib outputs log into a file.

Finally I call this Ansible role

- name: Run dbscontrol utility
  dbscontrol:
    user: "{{ hostvars[groups['dbs_server'][0]]['ansible_user'] }}"
    host: "{{ groups['dbs_server'][0] }}"
    script: "dbscontrol_config.yml"
  register: result
- debug:
    msg: "{{ result }}"

From the last logger message in my lib I can clearly see that the run completed successfully however my module ends up failing with massive output from the logger messages that starts with

MSG:

MODULE FAILURE
See stdout/stderr for the exact error

Oddly enough I see result embedded into MODULE_STDOUT section of the output. In fact it's the last section before MODULE_STDERR starts

Both MODULE_STDOUT and MODULE_STDERR consist of identical logging messages from the lib with the only difference of result related lines:

2020-01-23 13:40:52,070 - ttautils.dbscontrol - INFO - DBS control run is complete, exiting

{"changed": false, "original_message": "Running DBSCONTROL", "message": "DBSCONTROL_SUCCESSFULL", 
    "invocation": {"module_args": {"user": "root", "host": "fiesta1.td.teradata.com", "script": 
    "dbscontrol_config.yml", "dbc_user": "dbc", "dbc_pwd": "dbc", "logfile": "dbscntl_2020-01-23-13-38-15.log",  
    "loglevel": "DEBUG", "validate": "False", "config": "user_config/common", "locale": "TPG_6700C", 
    "timeout": "7200", "disable_local_overrides": false, "params": "user_config/user.yml"}}, 
    "warnings": ["The value False (type bool) in a string field was converted to 'False' (type string). 
        If this does not look like what you expect, quote the entire value to ensure it does not change."]}

The problem is that module always ends up as "failed" and the playbook terminates even I know that my code ran successfully.

2 days later

Ok, I know the problem now. It's due to the library I'm wrapping writing output to STDOUT/STDERR since it's using subprocess internally. When Ansible tries to parse the STDOUT it fails in this method because of all the extra non-JSON output in the STDOUT. How to deal with this situation? How can I possibly guarantee that my custom module has a pristine STDOUT with only JSON-formatted output? I was trying to do sys.stdout.flush() to no avail.

This practically renders writing a custom module useless. Please Ansible gurus, any hints?

score 0 · Accepted Answer · answered Jan 27 '20 at 21:14

So the answer to this problem is simple: when you call exit_json or fail_json you can't have sys.stdout with any sort of output. The reason is also simple: after your code is ran the result is processed by this Ansible code. Which parses res.get('stdout', u'') and if there are any parsing errors then you end up with that massive STDOUT dump I mentioned in my question.

The solution is not that simple. I tried various hacks to empty STDOUT before calling Module exit methods but it didn't really worked. So since I have a full control over what I'm calling I simply changed my logging to temp-files based in the library I call and did few other things to ensure that I'm not using sys.stdout in any way. Which is really silly and I hope that someone can show me a better way

Ansible - dealing with unexpected STDOUT in custom module

1 Answers1