Im using ansible to provision multiple instance over aws or other cloud provisioner. This is done with the python API ansible provide.
If a run playbooks in a sequential manner it works fine. By sequential i mean, execute one at a time, only go to the next one when the previous is completed.
But i want to perform this in a asynchronous form. I want to execute several playbooks at the same time. Creating multiple instance at the same time.
When trying to run at least 2 or 3 at the same time it crashes with the following error:
File "/home/usr/proj/ansible.py", line 18, in __init__
self.inventory = InventoryManager(loader=self.loader, sources='/etc/ansible/hosts')
File "/home/usr/proj/venv/lib/python3.6/site-packages/ansible/inventory/manager.py", line 146, in __init__
self.parse_sources(cache=True)
File "/home/usr/proj/venv/lib/python3.6/site-packages/ansible/inventory/manager.py", line 207, in parse_sources
parse = self.parse_source(source, cache=cache)
File "/home/usr/proj/venv/lib/python3.6/site-packages/ansible/inventory/manager.py", line 253, in parse_source
for plugin in self._fetch_inventory_plugins():
File "/home/user/proj/venv/lib/python3.6/site-packages/ansible/inventory/manager.py", line 186, in _fetch_inventory_plugins
plugin = inventory_loader.get(name)
File "/home/usr/proj/venv/lib/python3.6/site-packages/ansible/plugins/loader.py", line 562, in get
obj = getattr(self._module_cache[path], self.class_name)
AttributeError: module 'ansible.plugins.inventory.script' has no attribute 'InventoryModule'
Other error that appears is the skipping of several plugins, but not sure if its related to the previous. Small example from the huge list:
[WARNING]: Skipping plugin (/home/usr/venv/lib/python3.6/site- packages/ansible/plugins/connection/buildah.py) as it seems to be invalid: module 'ansible.plugins.connection./home/usr/venv/lib/python3.6/site- packages/ansible/plugins/connection/buildah' has no attribute 'Connection'
The code from my app where the inventory is declared is:
def __init__(self):
self.loader = DataLoader()
self.inventory = InventoryManager(loader=self.loader, sources='/etc/ansible/hosts')
self.variable_manager = VariableManager(loader=self.loader, inventory=self.inventory)
def run_playbook(self, playbook_path: str, playbook_name: str, extra_vars: dict = None, callback=None):
variable_manager = VariableManager(loader=self.loader, inventory=self.inventory)
password = open("pass", "r").read()
variable_manager._extra_vars = {"ansible_sudo_pass": password}
variable_manager._extra_vars = {**variable_manager.extra_vars, **extra_vars}
playbook = playbook_path + playbook_name
if not os.path.exists(playbook_path):
print('[INFO] The playbook does not exist')
sys.exit()
# List Of Obligatory Flags To Pass When Running The Playbook
context.CLIARGS = ImmutableDict(listtags=False,
listtasks=False,
listhosts=False,
syntax=False,
start_at_task=None,
connection='ssh',
module_path=None,
forks=100,
remote_user='root',
private_key_file=None,
ssh_common_args='-o StrictHostKeyChecking=no',
ssh_extra_args='-o StrictHostKeyChecking=no',
sftp_extra_args=None,
scp_extra_args=None,
become=True,
become_method='sudo',
become_user='root',
verbosity=5,
check=False,
diff=False,
force_valid_group_names='never')
pbex = PlaybookExecutor(playbooks=[playbook],
inventory=self.inventory,
variable_manager=variable_manager,
loader=self.loader,
passwords={})
pbex._tqm._stdout_callback = callback
return pbex.run()
Tried both sources='localhost,' and the direct path to the hosts inventory
Ansble version is 2.8 and OS is Ubuntu 18
The code that i use is just a while with threads calling a rest endpoint that will trigger the execution of the playbook meanning the code above:
i = 0
while i <= 2:
x = threading.Thread(target=thread_func, args=(str(i),))
x.start()
i += 1
