Why does writing to file change the content (hash)?

Question

I noticed, that when trying to generate some hashes after copy-pasting them into a file with vim, the hash is not as it is supposed to be. Same when file was opened and written out. Same behavior with nano, so there must be something I am mising.

$ echo -n  "foo" | openssl dgst -sha256
2c26...e7ae
$ echo -n "foo" > hash.txt
$ openssl dgst -sha256 hash.txt
SHA256(hash.txt)= 2c26...e7ae

But when I open hash.txt with nano or vim and quit without inserting anything, I subsequently get the following hash: b5bb...944c

I also noticed that without opening the file and writing out I do not see the output when using cat or head. Was the encoding changed?

Somehow you inserted a newline into the file. In any event, I cannot reproduce your results. — President James K. Polk, May 26 '20 at 17:00
`echo foo | openssl dgst -sha256` => b5bb…4944c. This is the same `foo` with a newline. — phd, May 26 '20 at 18:05

score 4 · Accepted Answer · answered May 27 '20 at 01:05

Most text editors, including Vim, save the file with a newline at the end when you quit. That's because according to POSIX, a text file is either empty or ends with a newline, so most users expect and want that behavior.

As you've noticed, adding a newline at the end changes the hash, and the hash you're getting is consistent with the sequence foo\n, where \n is a newline.

If you have a recent enough Vim, you can control the behavior mentioned above by setting nofixeol, which will preserve the existing line ending, or lack thereof, on the last line.

I believe without `nofixeol`, `noeol` is documented to work only with `binary`. — bk2204, May 29 '20 at 21:08

Why does writing to file change the content (hash)?

1 Answers1