Ansible lineinfile not copying last line to server

Question

I am trying to copy a secret Python settings file from a settings server to the production environment. Since the settings contains passwords I am using Ansible Vault.

My playbook looks like this:

---
- hosts: production
  tasks:
  - include_vars: settings.yml
  - name: Set properties
    lineinfile:
      dest: ~/temp/deploy
      regexp: "{{ item.split('=')[0] }}\\s*="
      line: "{{ item }}"
    with_lines: echo "{{ config }}"

And my settings.yml looks like this:

config:  |
  ASD='DEF'
  PROGRAM='PROG'
  PASSWORD='MAGNUS123'
  TEMP='TEST'

However when I run the playbook I get the file:

ASD='DEF'
PROGRAM='PROG'
PASSWORD='MAGNUS123'

Even though Ansible claims the last line is also copied:

changed: [ssh.pythonanywhere.com] => (item=ASD='DEF' )
changed: [ssh.pythonanywhere.com] => (item=PROGRAM='PROG')
changed: [ssh.pythonanywhere.com] => (item=PASSWORD='MAGNUS123')
changed: [ssh.pythonanywhere.com] => (item=TEMP='TEST')
changed: [ssh.pythonanywhere.com] => (item=)

What am I doing wrong to cause this?

Ansible version:

ansible --version
ansible 2.4.1.0
  config file = None
  ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.6.2 (default, Jul 17 2017, 16:44:45) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]

I'm curious: did you invent this yourself or found somewhere? — techraf, Nov 16 '17 at 08:39
Invented it myself, if someone has a solution along the lines of "Scrap this and use this" I'll be all ears — Magnus Lundberg, Nov 16 '17 at 08:42
I have a python settings file on a server encrypted with Ansible Vault. I want to update the settings file on the production environment so that if the setting is present, it is updated, if it's not present it is appended to the file. — Magnus Lundberg, Nov 16 '17 at 08:51

techraf · Accepted Answer · 2017-11-16T11:05:59.123

You are doing something completely unnecessary (skip to the bottom), but let's give it a try...

What am I doing wrong to cause this?

If you run the following:

- shell: echo "{{ config }}" | hexdump
  register: echo

- debug:
    var: echo.stdout

You'll see that the output ends with:

54 45 4d 50 3d 27 54 45 53 54 27 0a 0a

which is TEMP='TEST' followed by two newline characters. When used in with_lines loop this will cause two iterations -- the last one with an empty value passed to item.

Now, think what happens to your arguments when item is empty:

regexp becomes \\s*= and line is empty.

In other words you instruct Ansible to replace some line containing = with an empty line.

If the starting point is an empty file, Ansible does the following:

Adds ASD='DEF' line.
Adds PROGRAM='PROG' line.
Adds PASSWORD='MAGNUS123' line.
Adds TEMP='TEST' line.
Replaces a line with = with an empty line (in fact it happens to be the very last line).

And that is the result you get: three lines and an empty one.

^{You can also arrive at the same conclusion simply running using debug module to display the values of "{{ item.split('=')[0] }}\\s*=" and {{ item }} which you pass to the lineinfile arguments.}

echo has -n argument ("Do not print the trailing newline character"), but for a reason beyond my current understanding, it does not change the result (check with hexdump).

If you however replaced echo with printf -- the dangling 0x0a is not there and you get expected result (check with hexdump).

All this, apart from being a nice conundrum, is yet another argument against using linefile module in Ansible.

Use copy (you can protect the whole file with Ansible Vault - see decrypt argument) or template (you can have a public template and store the variable values in a Vault-protected form). Define the desired state, do not rely on current state. Period.

That you for both telling me what was wrong (that worked perfectly) and what I should use instead — Magnus Lundberg, Nov 16 '17 at 10:56

Ansible lineinfile not copying last line to server

1 Answers1