hash each line in text file

Question

I'm trying to write a little script which will open a text file and give me an md5 hash for each line of text. For example I have a file with:

123
213
312

I want output to be:

ba1f2511fc30423bdbb183fe33f3dd0f
6f36dfd82a1b64f668d9957ad81199ff
390d29f732f024a4ebd58645781dfa5a

I'm trying to do this part in bash which will read each line:

#!/bin/bash
#read.file.line.by.line.sh

while read line
do
echo $line
done

later on I do:

$ more 123.txt | ./read.line.by.line.sh | md5sum | cut -d '  ' -f 1

but I'm missing something here, does not work :(

Maybe there is an easier way...

score 37 · Answer 1 · edited Feb 23 '18 at 12:23

37

Almost there, try this:

while read -r line; do printf %s "$line" | md5sum | cut -f1 -d' '; done < 123.txt

Unless you also want to hash the newline character in every line you should use printf or echo -n instead of echo option.

In a script:

#! /bin/bash
cat "$@" | while read -r line; do
    printf %s "$line" | md5sum | cut -f1 -d' '
done

The script can be called with multiple files as parameters.

edited Feb 23 '18 at 12:23

Socowi

answered May 04 '11 at 21:42

Lars Wiegman

2

Or pass the string to md5's stdin without "echo": md5 <<< "$line" – glenn jackman May 04 '11 at 23:43
4

@glenn jackman Your example will also hash the newline character of each line which changes the hash of the string. – Lars Wiegman May 05 '11 at 08:10

score 6 · Answer 2 · answered May 04 '11 at 21:38

6

You can just call md5sum directly in the script:

#!/bin/bash
#read.file.line.by.line.sh

while read line
do
echo $line | md5sum | awk '{print $1}'
done

That way the script spits out directly what you want: the md5 hash of each line.

answered May 04 '11 at 21:38

Wes Hardaker

6

You should probably put the awk outside loop, to avoid spawning an unnecessary instance of it for each line. Just do `done | awk '{print $1}'`. – John Zwinck May 04 '11 at 21:43
6

You should do echo -n $line because by default, echo adds a newline to the end of the string. Consequently, it changes the MD5 value of the string. – Hai Vu May 05 '11 at 07:25
2

His example included the newline in the hash... I checked the values. – Wes Hardaker May 05 '11 at 13:56

score 0 · Answer 3 · answered Dec 06 '21 at 13:14

0

this worked for me..

cat $file | while read line; do printf %s "$line" | tr -d '\r\n' | md5 >> hashes.csv; done

answered Dec 06 '21 at 13:14

1

Your file had DOS terminated lines, which is a different problem. The correct way would have been: `sed 's/^M$//' "$file" | while read -r line; do printf %s "$line" | md5; done >> hashes.txt` – Fravadona Dec 06 '21 at 13:22

3 Answers3