Ansible concatenate 2 files with jinja2 template

Question

I am trying to vertically concatenate 2 files

The 1st file is roles/myrolename/files/requirements.dat

numpy
matplotlib
dask

and the 2nd file is ~/myprojects/myprojectname/requirements.txt

django
pywinrm
scipy

Here is my attempt to combine these 2 files, based on

Here is the task I am using

- name: Play to combine 2 files
  hosts: localhost
  connection: local

  vars:
    myprojects_path: myprojects
    my_proj_name: myprojectname

  tasks:
    - name: Concatenate 2 files
      template:
        src: "roles/myrolename/templates/requirements.dat.j2"
        dest: "{{ myprojects_path }}/{{ my_proj_name }}/requirements.dat"

Here is roles/myrolename/templates/requirements.dat.j2

{% include 'roles/myrolename/files/requirements.dat' %}
{% include myprojects_path + "/" + my_proj_name + "/" + "requirements.txt" %}

This task gives the error message (related to the 2nd include statement from roles/myrolename/templates/requirements.dat.j2)

fatal: [my_local_machine]: FAILED! => {"changed": false, 
"msg": "TemplateNotFound: /home/wr/myprojects/myprojectname/requirements.txt"}

This seems to suggest that Ansible is expecting that /home/wr/myprojects/myprojectname/requirements.txt is a template and not a file. However, the first include line does not throw an error. This is confusing me.

Is there a way I can use include to concatenate these 2 files with the jinja2 template, or is there another method to concatenate the 2 files?

EDIT 1:

I added {# a comment #} to the 1st line of requirements.txt. Here is the new ~/myprojects/myprojectname/requirements.txt

{# a comment #}
django
pywinrm
scipy

Then, I re-ran the playbook ansible-playbook test.yml -vvv. Here is the output

<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: wr
<127.0.0.1> EXEC /bin/sh -c 'echo ~wr && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/wr/.ansible/tmp/ansible-tmp-1532962816.77-184726406959930 `" && echo ansible-tmp-1532962816.77-184726406959930="` echo /home/wr/.ansible/tmp/ansible-tmp-1532962816.77-184726406959930 `" ) && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/wr/.ansible/tmp/ansible-tmp-1532962816.77-184726406959930/ > /dev/null 2>&1 && sleep 0'
fatal: [wr-box]: FAILED! => {
    "changed": false, 
    "msg": "TemplateNotFound: /home/wr/myprojects/myprojectname/requirements.txt"
}

EDIT 2:

I tried to ask Ansible to treat the above requirements.txt (with {# a comment #} on 1st line) as a jinja2 template by re-naming the file. I renamed the file requirements.txt to requirements.txt.j2 and re-ran the playbook. It gives this output

<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: wr
<127.0.0.1> EXEC /bin/sh -c 'echo ~wr && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/wr/.ansible/tmp/ansible-tmp-1532963056.03-76693523230507 `" && echo ansible-tmp-1532963056.03-76693523230507="` echo /home/wr/.ansible/tmp/ansible-tmp-1532963056.03-76693523230507 `" ) && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /home/wr/.ansible/tmp/ansible-tmp-1532963056.03-76693523230507/ > /dev/null 2>&1 && sleep 0'
fatal: [wr-box]: FAILED! => {
    "changed": false, 
    "msg": "TemplateNotFound: /home/wr/myprojects/myprojectname/requirements.txt.j2"
}

It still does not seem to find the template file, even though I have checked that it is indeed present at the location I specified.

Have you tried to run in verbose mode to see if there is an hint how jinja was resolving the inclusion ? Have you tried to put a templating instruction in `requirement.txt` like `{# a comment #}`. — Baptiste Mille-Mathias, Jul 30 '18 at 06:13
https://github.com/ansible/ansible/issues/7106#issuecomment-127825573 — techraf, Jul 30 '18 at 15:24
@techraf Thank you. My `requirements.txt` file was located outside the playbook directory. As your link suggests, this is not allowed so it was not detecting the `requirements.txt` as a template. This answers my question. I will create templates in `templates/` only and call them from there. — edesz, Jul 30 '18 at 15:28

score 5 · Accepted Answer · answered Jul 30 '18 at 06:42

5

Generally the include statement treats the file as a template. Also the file requirements.txt in your example is generally a template, without variables and with text only.

Does the file /home/wr/myprojects/myprojectname/requirements.txt exist?

answered Jul 30 '18 at 06:42

Vladimir Botka

58,131
4
32
63

Yes, that file does exist. But, as you say, it is a text file without variables. – edesz Jul 30 '18 at 14:52

Ansible concatenate 2 files with jinja2 template

1 Answers1