Bash loop to compare files

Question

I'm obviously missing something simply, and know the problem is that it's creating a blank output which is why it can't compare. However if someone could shed some light on this it would be great - I haven't isolated it.

Ultimately, I'm trying to compare the md5sum from a list stored in a txt file, to that stored on the server. If errors, I need it to report that. Here's the output:

root@vps [~/testinggrounds]# cat md5.txt | while read a b; do
>   md5sum "$b" | read c d
>   if [ "$a" != "$c" ] ; then
>     echo "md5 of file $b does not match"
>   fi
> done
md5 of file file1 does not match
md5 of file file2 does not match

root@vps [~/testinggrounds]# md5sum file*
2a53da1a6fbfc0bafdd96b0a2ea29515  file1
bcb35cddc47f3df844ff26e9e2167c96  file2

root@vps [~/testinggrounds]# cat md5.txt
2a53da1a6fbfc0bafdd96b0a2ea29515  file1
bcb35cddc47f3df844ff26e9e2167c96  file2

score 7 · Answer 1 · answered Sep 13 '12 at 19:27

7

Not directly answering your question, but md5sum(1):

-c, --check
read MD5 sums from the FILEs and check them

Like:

$ ls
1.txt  2.txt  md5.txt
$ cat md5.txt
d3b07384d113edec49eaa6238ad5ff00  1.txt
c157a79031e1c40f85931829bc5fc552  2.txt
$ md5sum -c md5.txt
1.txt: OK
2.txt: OK

answered Sep 13 '12 at 19:27

themel

8,825
2
32
31

This is probably the best route, but I am betting on not all versions of md5sum support this. The question is tagged linux though. – jordanm Sep 13 '12 at 19:33

jordanm · Answer 2 · 2012-09-13T20:48:02.723

4

The problem that you are having is that your inner read is executed in a subshell. In bash, a subshell is created when you pipe a command. Once the subshell exits, the variables $c and $d are gone. You can use process substitution to avoid the subshell:

while read -r -u3 sum filename; do
   read -r cursum _ < <(md5sum "$filename")
   if [[ $sum != $cursum ]]; then
      printf 'md5 of file %s does not match\n' "$filename"
   fi
done 3<md5.txt

The redirection 3<md5.txt causes the file to be opened as file descriptor 3. The -u 3 option to read causes it to read from that file descriptor. The inner read still reads from stdin.

edited Sep 13 '12 at 20:48

answered Sep 13 '12 at 19:23

jordanm

33,009
7
61
76

The inner `read` command isn't inheriting stdin from the enclosing while loop; it's connected to `md5sum` via the pipe. You're solving the problem by using process substitution (instead of a pipe) on the inner read, not by using a separate file descriptor for the outer loop. – chepner Sep 13 '12 at 19:59

score 3 · Accepted Answer · answered Sep 13 '12 at 21:14

I'm not going to argue. I simply try to avoid double read from inside loops.

#! /bin/bash

cat md5.txt | while read sum file
do
    prev_sum=$(md5sum $file | awk '{print $1}')
    if [ "$sum" != "$prev_sum" ]
    then
        echo "md5 of file $file does not match"
    else
        echo "$file is fine"
    fi
done

Bash loop to compare files

3 Answers3

Linked